search

ipfs / go-ds-flatfs

A datastore implementation using sharded directories and flat files to store data
MIT License
46 stars 21 forks source link

File mode on windows causes issues between renaming and writes #70

Open willscott opened 4 years ago

willscott commented 4 years ago

ref: https://github.com/ipfs/go-ipfs/issues/7115#event-3219629851

Current attempt towards mitigation is to open files on windows with the SHARE_DELETE flag per https://github.com/golang/go/issues/32088

willscott commented 4 years ago

After the change in #71, the following windows errors occur in the test suite:

--- FAIL: TestMoveRestart (0.96s)
    convert_test.go:92: Could not undo the move. C:\Users\circleci\AppData\Local\Temp\test-datastore-flatfs-525984286\v1: SHARDING file not found in datastore
    flatfs_test.go:34: tempdir cleanup failed: remove C:\Users\circleci\AppData\Local\Temp\test-datastore-flatfs-525984286\v1\XTV: Access is denied.
--- FAIL: TestSuite (84.81s)
    --- FAIL: TestSuite/github.com/ipfs/go-datastore/test.SubtestCombinations (77.27s)
        --- FAIL: TestSuite/github.com/ipfs/go-datastore/test.SubtestCombinations/1/{SELECT·keys,vals·FILTER·[KEY·<·"/2"]·ORDER·[KEY]} (0.00s)
            basic_tests.go:430: putting 4 values
            basic_tests.go:438: getting values back
            basic_tests.go:450: querying values
            basic_tests.go:460: aggregating query results
            basic_tests.go:463: query result error:  walk failed: open C:\Users\circleci\AppData\Local\Temp\test-datastore-flatfs-599044958\diskUsage.cache: The process cannot access the file because it is being used by another process.
        --- FAIL: TestSuite/github.com/ipfs/go-datastore/test.SubtestCombinations/1/{SELECT·keys,vals·FILTER·[KEY·<·"/2"]·ORDER·[KEY]·OFFSET·95·LIMIT·1} (0.00s)
            basic_tests.go:430: putting 4 values
            basic_tests.go:438: getting values back
            basic_tests.go:450: querying values
            basic_tests.go:460: aggregating query results
            basic_tests.go:463: query result error:  walk failed: open C:\Users\circleci\AppData\Local\Temp\test-datastore-flatfs-599044958\diskUsage.cache: The process cannot access the file because it is being used by another process.
        --- FAIL: TestSuite/github.com/ipfs/go-datastore/test.SubtestCombinations/1/{SELECT·keys,vals·FILTER·[KEY·<·"/2"]·ORDER·[FN]·OFFSET·95·LIMIT·10} (0.00s)
            basic_tests.go:430: putting 4 values
            basic_tests.go:438: getting values back
            basic_tests.go:450: querying values
            basic_tests.go:460: aggregating query results
            basic_tests.go:463: query result error:  walk failed: open C:\Users\circleci\AppData\Local\Temp\test-datastore-flatfs-599044958\diskUsage.cache: The process cannot access the file because it is being used by another process.

(ci run: https://app.circleci.com/pipelines/github/ipfs/go-ds-flatfs/48/workflows/c2956bfa-5314-4106-baf4-932fab6eae0b/jobs/126)