syndicate-storage / syndicate

Internet-scale software-defined storage system
Apache License 2.0
56 stars 10 forks source link

File creation bug in sub directory #25

Closed iychoi closed 11 years ago

iychoi commented 11 years ago

Hi.

I met a bug while creating a new file in sub directory. I could replay the same problem with the syndicatefs (UG).

I used local MS, so probably this might be only replayable with local MS. After create a directory, just create a new file with "touch" command of linux. This causes infinite loop in uploader thread of ms client.

I don't know why it couldn't succeed uploading.

syndicatefs console shows below

... [build/out/libsyndicate/ms-client.cpp:0363] ms_client_uploader_thread: 0x9fc300 sync'ed updates, rc = -2 [build/out/libsyndicate/ms-client.cpp:0366] ms_client_uploader_thread: WARN: ms_entry_sync_updates rc = -2 [build/out/libsyndicate/ms-client.cpp:0344] ms_client_uploader_thread: 1 pending deadlines [build/out/libsyndicate/ms-client.cpp:0846] update_set_iterator: update(path=/master/abc.ttt, url=http://iychoi-virtual-machine:32780/SYNDICATE-DATA///master/abc.ttt.1373606182331) DATA MS send 0.082964 DATA X-Volume-Time 0.000000 DATA X-UG-Time 0.000000 DATA X-Total-Time 0.000000 ...

And, GAE console shows below

... INFO 2013-07-12 05:16:36,060 entry.py:589] update '/master/abc.ttt' INFO 2013-07-12 05:16:36,109 recording.py:665] Saved; key: appstats:096000, part: 115 bytes, full: 20868 bytes, overhead: 0.001 + 0.013; link: http://localhost:8080/_ah/stats/details?time=1373606196027 INFO 2013-07-12 05:16:36,118 server.py:593] default: "POST /FILE/testvolume-iychoi-email.arizona.edu/ HTTP/1.1" 202 3 INFO 2013-07-12 05:16:36,161 entry.py:589] update '/master/abc.ttt' INFO 2013-07-12 05:16:36,199 recording.py:665] Saved; key: appstats:096100, part: 115 bytes, full: 20851 bytes, overhead: 0.004 + 0.010; link: http://localhost:8080/_ah/stats/details?time=1373606196131 INFO 2013-07-12 05:16:36,219 server.py:593] default: "POST /FILE/testvolume-iychoi-email.arizona.edu/ HTTP/1.1" 202 3 INFO 2013-07-12 05:16:36,269 entry.py:589] update '/master/abc.ttt' INFO 2013-07-12 05:16:36,309 recording.py:665] Saved; key: appstats:096200, part: 115 bytes, full: 20836 bytes, overhead: 0.001 + 0.022; link: http://localhost:8080/_ah/stats/details?time=1373606196227 INFO 2013-07-12 05:16:36,317 server.py:593] default: "POST /FILE/testvolume-iychoi-email.arizona.edu/ HTTP/1.1" 202 3 ...

"master" in this message is directory and "abc.ttt" is file I created.

However, creating sub directory in another directory works.

Here's my command.

iychoi@iychoi-virtual-machine:/mnt/syndicatefs$ ls iychoi@iychoi-virtual-machine:/mnt/syndicatefs$ mkdir master iychoi@iychoi-virtual-machine:/mnt/syndicatefs$ ls master iychoi@iychoi-virtual-machine:/mnt/syndicatefs$ ls -al total 5 drwxrwxrwx 1 games tty 0 Jul 11 22:14 . drwxr-xr-x 3 root root 4096 Jun 4 13:32 .. drwxrwxr-x 1 games tty 4096 Jul 11 22:14 master iychoi@iychoi-virtual-machine:/mnt/syndicatefs$ touch abc.txt iychoi@iychoi-virtual-machine:/mnt/syndicatefs$ ls abc.txt master iychoi@iychoi-virtual-machine:/mnt/syndicatefs$ cd master/ iychoi@iychoi-virtual-machine:/mnt/syndicatefs/master$ ls iychoi@iychoi-virtual-machine:/mnt/syndicatefs/master$ touch abc.ttt touch: closing `abc.ttt': No such file or directory iychoi@iychoi-virtual-machine:/mnt/syndicatefs/master$ ls abc.ttt iychoi@iychoi-virtual-machine:/mnt/syndicatefs/master$

Thank you for your time.

iychoi commented 11 years ago

Problem solved after git pull.

Below is Jude's message.

The problem was that the file creation was being delayed due to the write TTL of the parent directory not being 0 (it's 5000 by default). Not only was this not being handled properly (i.e. the CREATE message was never sent to the MS), but also creating files asynchronously with the MS will not be supported (at least, not until there's a compelling use-case for it).