fsouza / fake-gcs-server

Google Cloud Storage emulator & testing library.
https://pkg.go.dev/github.com/fsouza/fake-gcs-server/fakestorage?tab=doc
BSD 2-Clause "Simplified" License
1.03k stars 209 forks source link

Support for generations in the filesystem backend #744

Open fsouza opened 2 years ago

fsouza commented 2 years ago

Currently, only the memory backend supports generations, we should implement generations in the filesystem backend too.

One thing to notice is that while we don't have any promises in terms backwards compatibility for the contents of -filesystem-root, maybe we shouldn't break it too badly? Meaning, we'd probably want to introduce a new backend, or some migration script (I doubt users would notice the migration script until too late? idk).

orsinium commented 2 years ago

There is a Python script using dimutex that reproduces the issue:

import asyncio
import aiohttp
import dimutex
from http import HTTPStatus
from gcloud.aio.storage.storage import API_ROOT
from random import choice
from string import ascii_letters

async def main():
    # create bucket
    bucket_name = "".join(choice(ascii_letters) for _ in range(10))
    connector = aiohttp.TCPConnector(ssl=False)
    async with aiohttp.ClientSession(connector=connector) as session:
        await session.post(
            url=API_ROOT,
            json=dict(name=bucket_name),

        )

    lock = dimutex.GCS(bucket=bucket_name, name='lock-name', required=False)
    async with lock:
        resp = await lock._create(force=False)
        resp.raise_for_status()
        resp = await lock._create(force=False)
        assert resp.status == HTTPStatus.PRECONDITION_FAILED

asyncio.run(main())

The assertion fails on #735 but passes without it. Using -backend memory fixes the issue.