Deploy www.calbatwg.org to Fly

[simonw]

Split from:

• simonw/household#26

Code is in https://github.com/natbat/cbwg

[simonw]

I signed in to Natalie's Fly account on my machine, and added my Amex as payment details.

Created a new PostgreSQL cluster.

% fly postgres create
? Choose an app name (leave blank to generate one): cbwg-postgresql
automatically selected personal organization: Natalie Downe
? Select region: San Jose, California (US) (sjc)
For pricing information visit: https://fly.io/docs/about/pricing/#postgresql-clu? Select configuration: Development - Single node, 1x shared CPU, 256MB RAM, 1GB disk
Creating postgres cluster in organization personal
Creating app...
Setting secrets on app cbwg-postgresql...
Provisioning 1 of 1 machines with image flyio/postgres:14.4

It gave me a credential string which I saved as a secure note in 1Password.

[simonw]

Problems to solve:

• [ ] Docker image that installs using pipenv
• [ ] Get a local build of that working
• [ ] Run migrations on startup (like pillarpointstewards does)
• [ ] What does django-heroku do? Do I need to replace it?

[simonw]

import django_heroku

django_heroku.settings(locals())

It's being sunset for good reasons:

• https://github.com/heroku/django-heroku/issues/56

https://github.com/heroku/python-getting-started/commit/14230b2be293b5a94deb8c7caeb8c7727adb9ec0 is an example showing how to replace it. Key bits:

import dj_database_url

BASE_DIR = Path(__file__).resolve().parent.parent
IS_HEROKU = "DYNO" in os.environ

if 'SECRET_KEY' in os.environ:
    SECRET_KEY = os.environ["SECRET_KEY"]

if IS_HEROKU:
    ALLOWED_HOSTS = ["*"]
else:
    ALLOWED_HOSTS = []

if "DATABASE_URL" in os.environ:
    # Configure Django for DATABASE_URL environment variable.
    DATABASES["default"] = dj_database_url.config(
        conn_max_age=600, ssl_require=True)

STATIC_ROOT = BASE_DIR / "staticfiles"
STATIC_URL = "static/"

# Enable WhiteNoise's GZip compression of static assets.
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"

Also adds this to middleware:

    "whitenoise.middleware.WhiteNoiseMiddleware",

[simonw]

Trying this Dockerfile as a starting point:

FROM python:3.11.0-slim-bullseye

RUN mkdir -p /app
WORKDIR /app

COPY Pipfile.lock .
RUN pip install pipenv
RUN pipenv sync

Put that in the cbwg/ directory and ran:

docker build . -t cbwg

[simonw]

Got a ton of installation errors that looked like this:

simonw/household#10 16.21 An error occurred while installing boto3==1.7.4 --hash=sha256:13ad5f64a247d655a27dca83274588e9d14cba61b38d3d4fd2b011b7197d88dd --hash=sha256:a56b21efbc994580fc9cef454f0f949745c152326f939aed6609d1c47b2a0f8f! Will try again.

Problem with the Pipfile.lock - I'll try regenerating that perhaps?

https://stackoverflow.com/questions/56646378/install-from-pipfile-using-pipenv-install-gives-error/70793257#70793257 says:

It was the mismatch between my virtualenv python version (3.6) and my own system python version (3.8)

The Heroku app was on Python 3.6, I'll try that first.

[simonw]

Trying with:

FROM python:3.6.15-slim-buster

The build completed! Looks like I have a mechanism for installing the dependencies in Docker.

[simonw]

import django is failing when I try to run management commands.

docker run -it --rm --name cbwg -p 8000:8000 cbwg bash

Then:

root@a6e8ba485174:/app# python
Python 3.6.15 (default, Dec 21 2021, 12:28:02) 
...
>>> import django
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'django'

Running pipenv shell first in the /app directory fixes that.

[simonw]

Now I'm getting:

simonw/household#17 0.978 ModuleNotFoundError: No module named 'cbwg'

When I try to run the this line in the Dockerfile:

RUN pipenv run ./manage.py collectstatic --noinput

[simonw]

Changing the COPY lines to this fixed it - it looks like COPY cbwg . may have been copying the files within that directory directly into /app.

COPY manage.py manage.py
COPY blog blog
COPY cbwg cbwg
COPY home home
COPY media media
COPY search search

Here's the full Dockerfile which builds successfully:

FROM python:3.6.15-slim-buster

RUN mkdir -p /app
WORKDIR /app

COPY Pipfile.lock .
RUN pip install pipenv
RUN pipenv sync

COPY manage.py manage.py
COPY blog blog
COPY cbwg cbwg
COPY home home
COPY media media
COPY search search

RUN pipenv run ./manage.py collectstatic --noinput

EXPOSE 8000

CMD pipenv run gunicorn --bind :8000 --workers 2 cbwg.wsgi

[simonw]

Now going to try using this trick to connect to PostgreSQL: https://til.simonwillison.net/docker/docker-for-mac-container-to-postgresql-on-host

I connected to my template1 database in PostgreSQL running on my Mac (not in Docker) and ran:

create database cbwg;

I already had a docker role - I then connected to cwbg and ran this:

GRANT ALL ON ALL TABLES IN SCHEMA "public" TO docker;

Now trying this:

docker run -it --rm \
  --env SECRET_KEY=9f1b7ae47970b77a49fa6f7cfcf3cd6b \
  --env DJANGO_SETTINGS_MODULE="cbwg.settings.production" \
  --env DATABASE_URL="postgres://docker:docker-password@host.docker.internal:5432/cbwg" \
  --name cbwg -p 8000:8000 \
  cbwg

Hitting the homepage resulted in a timeout error:

[2022-11-27 18:40:38 +0000] [7] [CRITICAL] WORKER TIMEOUT (pid:15)

[simonw]

Maybe because of this in production.py:

SECURE_SSL_REDIRECT = True

Tried commenting that out and rebuilding but it didn't fix it.

[simonw]

Trying to poke around in bash a bit:

docker run -it --rm \
  --env SECRET_KEY=9f1b7ae47970b77a49fa6f7cfcf3cd6b \
  --env DJANGO_SETTINGS_MODULE="cbwg.settings.production" \
  --env DATABASE_URL="postgres://docker:docker-password@host.docker.internal:5432/cbwg" \
  --name cbwg -p 8000:8000 \
  cbwg bash

Got this:

root@b2817fbda76c:/app# pipenv run manage.py shell
Error: the command manage.py could not be found within PATH or Pipfile's [scripts].
root@b2817fbda76c:/app# pipenv run ./manage.py shell
Python 3.6.15 (default, Dec 21 2021, 12:28:02) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import django
>>> from django.contrib.auth.models import User
>>> User.objects.all()
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.6/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
    self.connect()
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.6/site-packages/django/db/backends/base/base.py", line 189, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: server does not support SSL, but SSL was required

[simonw]

I think django_heroku.settings(locals()) may be trying to enforce SSL connection to the database. I'll ditch that and try again.

[simonw]

These changes got it working - to the point where I got an error about the tables not being created yet:

diff --git a/cbwg/settings/production.py b/cbwg/settings/production.py
index bda7af2..8f4deeb 100644
--- a/cbwg/settings/production.py
+++ b/cbwg/settings/production.py
@@ -1,5 +1,5 @@
 from __future__ import absolute_import, unicode_literals
-import django_heroku
+import dj_database_url
 import os
 from .base import *

@@ -10,7 +10,15 @@ try:
 except ImportError:
     pass

-SECURE_SSL_REDIRECT = True
+ALLOWED_HOSTS = ["*"]
+
+SECRET_KEY = os.environ["SECRET_KEY"]
+
+if "DATABASE_URL" in os.environ:
+    # Configure Django for DATABASE_URL environment variable.
+    DATABASES["default"] = dj_database_url.config(conn_max_age=600)
+
+# SECURE_SSL_REDIRECT = True

 SENTRY_DSN = os.environ.get('SENTRY_DSN')
 if SENTRY_DSN:
@@ -21,6 +29,3 @@ if SENTRY_DSN:
         'dsn': SENTRY_DSN,
         'release': os.environ.get('HEROKU_SLUG_COMMIT', ''),
     }
-
-
-django_heroku.settings(locals())

[simonw]

Successfully ran the migrations:

docker run -it --rm \
  --env SECRET_KEY=9f1b7ae47970b77a49fa6f7cfcf3cd6b \
  --env DJANGO_SETTINGS_MODULE="cbwg.settings.production" \
  --env DATABASE_URL="postgres://docker:docker-password@host.docker.internal:5432/cbwg" \
  --name cbwg -p 8000:8000 \
  cbwg pipenv run ./manage.py migrate

[simonw]

Still getting a timeout error when I try to hit http://localhost:8000/ on my Mac.

But if I attach to the running container I can hit it from Python:

docker ps # to find the ID

docker exec -it 7263b95b42ef bash

python

import urllib.request
urllib.request.urlopen("http://localhost:8000/").read()

[simonw]

Those worker errors are unrelated to if the container is getting traffic or not:

[2022-11-27 20:58:33 +0000] [32] [INFO] Booting worker with pid: 32
[2022-11-27 20:58:33 +0000] [34] [INFO] Booting worker with pid: 34
[2022-11-27 21:09:02 +0000] [8] [CRITICAL] WORKER TIMEOUT (pid:32)
[2022-11-27 21:09:02 +0000] [8] [CRITICAL] WORKER TIMEOUT (pid:34)
[2022-11-27 21:09:02 +0000] [32] [INFO] Worker exiting (pid: 32)
[2022-11-27 21:09:02 +0000] [34] [INFO] Worker exiting (pid: 34)
[2022-11-27 21:09:02 +0000] [36] [INFO] Booting worker with pid: 36
[2022-11-27 21:09:03 +0000] [38] [INFO] Booting worker with pid: 38
[2022-11-27 21:12:41 +0000] [8] [CRITICAL] WORKER TIMEOUT (pid:36)
[2022-11-27 21:12:41 +0000] [8] [CRITICAL] WORKER TIMEOUT (pid:38)
[2022-11-27 21:12:41 +0000] [38] [INFO] Worker exiting (pid: 38)
[2022-11-27 21:12:41 +0000] [36] [INFO] Worker exiting (pid: 36)
[2022-11-27 21:12:41 +0000] [40] [INFO] Booting worker with pid: 40

[simonw]

Based on https://stackoverflow.com/questions/10855197/gunicorn-worker-timeout-error/24305939#24305939 I added --timeout 120 to the gunicorn line and the problem went away.

Also turns out I need to be absolutely sure to hit http://localhost:8000/ and not https://localhost:8000/.

[simonw]

It's not showing static assets yet though, need to fix that by setting up whitenoise.

[simonw]

Next step: deploy it to Fly.

[simonw]

Ran this:

flyctl apps create cbwg

Created fly.toml with this:

app = "cbwg"

kill_signal = "SIGINT"
kill_timeout = 5

[deploy]
  release_command = "/app/manage.py migrate --noinput"

[[services]]
  internal_port = 8000
  protocol = "tcp"

  [services.concurrency]
    hard_limit = 25
    soft_limit = 20

  [[services.ports]]
    handlers = ["http"]
    port = "80"

  [[services.ports]]
    handlers = ["tls", "http"]
    port = "443"

  [[services.tcp_checks]]
    interval = 10000
    timeout = 2000
    grace_period = "10s"

Now running:

fly deploy

[simonw]

image: registry.fly.io/cbwg:deployment-01GJXFPMCEVK4V1E1GFSEHCCA9
image size: 350 MB
==> Creating release
--> release v2 created

--> You can detach the terminal anytime without stopping the deployment
==> Release command detected: /app/manage.py migrate --noinput

--> This release will not be available until the release command succeeds.
     Starting instance
     Configuring virtual machine
     Pulling container image
     Unpacking image
     Preparing kernel init
     Configuring firecracker
       File "/app/manage.py", line 10, in <module>
         from django.core.management import execute_from_command_line
     ModuleNotFoundError: No module named 'django'
     Starting clean up.
Error release command failed, deployment aborted

It's because of my release command not using pipenv run I think.

[simonw]

This time got:

     django.db.utils.OperationalError: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
     Starting clean up.

Which makes sense because I haven't set DATABASE_URL yet.

Ran these:

fly secrets set DATABASE_URL=postgres://postgres:...@cbwg-postgresql.internal:5432
fly secrets set SECRET_KEY=...

[simonw]

Those gave me errors because the app isn't running OK yet. But I then tried:

fly deploy

And got this:

==> Creating release
--> release v2 created

--> You can detach the terminal anytime without stopping the deployment
==> Release command detected: pipenv run /app/manage.py migrate --noinput

--> This release will not be available until the release command succeeds.
Error release command failed, deployment aborted

Not sure what the error was this time though.

[simonw]

Tried running this too:

fly secrets set DJANGO_SETTINGS_MODULE="cbwg.settings.production"
fly deploy

Failed again. But this time fly logs showed me a different error:

2022-11-27T21:33:53Z app[dffc90c0] sjc [info]    "settings.DATABASES is improperly configured. "
2022-11-27T21:33:53Z app[dffc90c0] sjc [info]django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the NAME value.

[simonw]

I worried that zsh might have messed up my PostgreSQL connection string since I didn't wrap it in quotes, so I tried this:

cbwg % fly secrets set DATABASE_URL='postgres://postgres:...@cbwg-postgresql.internal:5432'
Error No change detected to secrets. Skipping release.

Looks like the connection string quoting made no difference.

[simonw]

I'm going to try removing that release command entirely so I can get a broken release out and SSH in to poke it.

[simonw]

With that line commented out:

image: registry.fly.io/cbwg:deployment-01GJXGD03EGM07226A0XH3ZV86
image size: 350 MB
==> Creating release
--> release v3 created

--> You can detach the terminal anytime without stopping the deployment
==> Monitoring deployment

 1 desired, 1 placed, 1 healthy, 0 unhealthy [health checks: 1 total, 1 passing]
--> v3 deployed successfully

[simonw]

https://cbwg.fly.dev/ is now live (and serving a 500 error).

[simonw]

OK, I can SSH in:

cbwg % fly ssh console
Connecting to fdaa:0:ec24:a7b:a15f:fe76:fd89:2... complete

But I cannot figure out how to use pipenv run to access the right virtual environment:

# cd /app
# pipenv run python ./manage.py shell
Creating a virtualenv for this project...
Pipfile: /app/Pipfile
Using /usr/local/bin/python3.6m (3.6.15) to create virtualenv...
⠸ Creating virtual environment...RuntimeError: failed to build image pip because:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py", line 55, in _install
    installer.install(creator.interpreter.version_info)
  File "/usr/local/lib/python3.6/site-packages/virtualenv/seed/embed/via_app_data/pip_install/base.py", line 31, in install
    self._uninstall_previous_version()
  File "/usr/local/lib/python3.6/site-packages/virtualenv/seed/embed/via_app_data/pip_install/base.py", line 142, in _uninstall_previous_version
    dist_name = self._dist_info.stem.split("-")[0]
  File "/usr/local/lib/python3.6/site-packages/virtualenv/seed/embed/via_app_data/pip_install/base.py", line 107, in _dist_info
    raise RuntimeError(f"no .dist-info at {self._image_dir}, has {', '.join(files)}")  # pragma: no cover
RuntimeError: no .dist-info at /.local/share/virtualenv/wheel/3.6/image/1/CopyPipInstall/pip-21.3.1-py3-none-any, has pip

✘ Failed creating virtual environment 
[pipenv.exceptions.VirtualenvCreationException]: 
Failed to create virtual environment.

[simonw]

I found the virtualenv by running this in /:

find . | grep django
# ...
./root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.6/site-packages/raven/contrib/django/__init__.py

[simonw]

This works:

/root/.local/share/virtualenvs/app-4PlAip0Q/bin/python3 manage.py shell

[simonw]

I tried this:

/root/.local/share/virtualenvs/app-4PlAip0Q/bin/python3 manage.py migrate

But got this traceback:

  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.6/site-packages/django/db/backends/base/base.py", line 188, in connect
    conn_params = self.get_connection_params()
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 158, in get_connection_params
    "settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the NAME value.

But running env confirms that the environment variables are set that should make this work:

FLY_PUBLIC_IP=2604:1380:45e1:3004:0:fe76:fd89:1
FLY_VM_MEMORY_MB=256
DATABASE_URL=postgres://postgres:...@cbwg-postgresql.internal:5432
SECRET_KEY=...
PYTHON_PIP_VERSION=21.2.4
HOME=/
OLDPWD=/root/.local/share/virtualenvs/app-4PlAip0Q
GPG_KEY=...
FLY_SSH=1
FLY_IMAGE_REF=registry.fly.io/cbwg@sha256:f2e7344881bcf7b8dbef33b3e9b6ecd7616af8df77f0e5f05a55def605c37ef4
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/3cb8888cc2869620f57d5d2da64da38f516078c7/public/get-pip.py
SSH_LISTEN=[fdaa:0:ec24:a7b:a15f:fe76:fd89:2]:22
TERM=xterm
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
FLY_ALLOC_ID=fe76fd89-72e8-44c7-0ba9-acf776305fce
DJANGO_SETTINGS_MODULE=cbwg.settings.production
LANG=C.UTF-8
cgroup_enable=memory
PYTHON_SETUPTOOLS_VERSION=57.5.0
PYTHON_VERSION=3.6.15
FLY_REGION=sjc
FLY_APP_NAME=cbwg
FLY_VCPU_COUNT=1
PWD=/app
PYTHON_GET_PIP_SHA256=c518250e91a70d7b20cceb15272209a4ded2a0c263ae5776f129e0d9b5674309
SSH_DNS_SERVER=[fdaa::3]:53

[simonw]

Tried this:

# /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python3 manage.py shell  
Python 3.6.15 (default, Dec 21 2021, 12:28:02) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.conf import settings
>>> settings.DATABASES
{'default': {'NAME': '', 'USER': 'postgres', 'PASSWORD': '...', 'HOST': 'cbwg-postgresql.internal', 'PORT': 5432, 'CONN_MAX_AGE': 600, 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'OPTIONS': {}, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}}
>>> 

Why is NAME an empty string there?

[simonw]

I don't think I've created a database yet! That connection string that Fly gave me looks like this:

postgres://postgres:...@cbwg-postgresql.internal:5432

But it should have /dbname at the end.

[simonw]

cbwg % fly postgres db list -a cbwg-postgresql
NAME        USERS                          
postgres    flypgadmin, postgres, repluser  

[simonw]

cbwg % fly postgres connect -a cbwg-postgresql
Connecting to fdaa:0:ec24:a7b:ad1:84:fd99:2... complete
psql (14.4 (Debian 14.4-1.pgdg110+1))
Type "help" for help.

postgres=# create database cbwg;
CREATE DATABASE

[simonw]

fly secrets set DATABASE_URL='postgres://postgres:...@cbwg-postgresql.internal:5432/cbwg'

[simonw]

And now this command worked:

/root/.local/share/virtualenvs/app-4PlAip0Q/bin/python3 /app/manage.py migrate

[simonw]

And https://cbwg.fly.dev/ works now!

[simonw]

Next steps:

• [x] Create a superuser account
• [ ] Use that to create regular user accounts
• [ ] Start copying over the data from the old site

[simonw]

# /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python3 /app/manage.py createsuperuser
Username (leave blank to use 'root'): admin
Email address: admin@example.com 
Password: 
Password (again): 
Superuser created successfully.

[simonw]

Got a 500 error uploading an image. I think it may be missing the S3 credentials it needs.

[simonw]

Found it:

AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME', '')
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID', '')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY', '')
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME

https://github.com/natbat/cbwg/blob/99ec903a4055067735dfab03003b7df1a2a8f606/cbwg/settings/base.py#L21-L24

[simonw]

I'm going to create a new bucket for this on my own personal AWS account - the bucket we use right now belongs to an AWS account we set up just for this project, but we no longer think that's a good idea because of the risk of losing access / forgetting to update billing / etc.

I'll call the new bucket calbatwg.

~ % s3-credentials create calbatwg --create-bucket
Created bucket: calbatwg
Created  user: 's3.read-write.calbatwg' with permissions boundary: 'arn:aws:iam::aws:policy/AmazonS3FullAccess'
Attached policy s3.read-write.calbatwg to user s3.read-write.calbatwg
Created access key for user: s3.read-write.calbatwg
{
    "UserName": "s3.read-write.calbatwg",
    "AccessKeyId": "AKIAWXFXAIOZA5PNQEVX",
    "Status": "Active",
    "SecretAccessKey": "...",
    "CreateDate": "2022-11-28 04:39:09+00:00"
}

[simonw]

fly secrets set AWS_STORAGE_BUCKET_NAME=calbatwg
fly secrets set AWS_ACCESS_KEY_ID=AKIAWXFXAIOZA5PNQEVX
fly secrets set AWS_SECRET_ACCESS_KEY='...'

[simonw]

That didn't fix it. I'm going to set up Sentry to see what the error is (nothing showed up in the logs).

fly secrets set SENTRY_DSN='https://...@....ingest.sentry.io/...'

[simonw]

Got this error in fly logs:

2022-11-28T04:48:16Z app[bb62a9ea] sjc [info]    raise InvalidDsn('Invalid Sentry DSN: %r' % url.geturl())
2022-11-28T04:48:16Z app[bb62a9ea] sjc [info]raven.exceptions.InvalidDsn: Invalid Sentry DSN: 'https://...@....ingest.sentry.io/...'

I guessed that this was maybe a new format and I needed an older one, so I set it to:

fly secrets set SENTRY_DSN=https://...:..@sentry.io/...

That fixed the exception. Now Sentry is showing me what the S3 error was:

[simonw]

I tested the credentials visible using env inside the container with Transmit and they worked fine for uploading files.

[simonw]

Could the problem be that https://cbwgsite.s3.amazonaws.com/images/ClimbersforBatConservation.63c572a5.fill-1000x400-c100.jpg is configured as a website, but the new bucket I created is not?