lock files are being created; but not always removed (was: binary files are not uploading)

[pkikawa]

When adding a new file using the web UI, lock files are being created but neither the binary file is uploaded nor any of it's associated files are created (with exception of lock file which is in an untracked state.

(ddrlocal) ddr@DDREditor:/media/qnfs/kinkura/gold/ddr-testing-40226$ git status
On branch master
Your branch is ahead of 'origin/master' by 16 commits.
(use "git push" to publish your local commits)
Untracked files:
(use "git add ..." to include in what will be committed)

    files/ddr-testing-40226-2/files/ddr-testing-40226-2-1/lock
    files/ddr-testing-40226-2/lock
nothing added to commit but untracked files present (use "git add" to track)

[pkikawa]

the gitstatus message at the top of the webpage only says `Pending: Uploading [example_binary] to [example_object]

[pkikawa]

attempting to delete binaries results in similar behavior.

(ddrlocal) ddr@DDREditor:/media/qnfs/kinkura/gold/ddr-testing-40212$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        lock

nothing added to commit but untracked files present (use "git add" to track)

[pkikawa]

I am wondering if this is a lock file issue because when i made a new collection it did not remove the lock file and the collection json had untracked changes and was not staged for commit

[GeoffFroh]

TODO: test again; but check and report the contents of the lock file

[pkikawa]

this is from creating an object OBJ_Creation_lock.zip

[pkikawa]

(ddrlocal) ddr@DDREditor:/media/qnfs/kinkura/gold/ddr-testing-40228$ git status
On branch master
Your branch is ahead of 'origin/master' by 6 commits.
  (use "git push" to publish your local commits)
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   files/ddr-testing-40228-23/entity.json

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        lock

no changes added to commit (use "git add" and/or "git commit -a")

[gjost]

I'm unable to duplicate this problem. I was able to successfully add and delete files to/from both entities and segments.

Is it possible that the collections where this was a problem contained untracked/unstaged files before the attempts? Is it possible that there are permissions issues in the collections?

[pkikawa]

This behavior is present even during the creation of a new collection, and the collections i've been testing in have all been clean.

as for permissions, everything looks normal

[pkikawa]

Sara has been able to replicate the lock file issue on her machine

[gjost]

OK you guys aren't crazy! :) It hasn't manifested itself in my local dev environment, but it looks like there is inconsistent use of locking in the file tasks. Some tasks lock at the collection level and some at the entity level. I'm going to change all file tasks to lock at the entity level (might actually be better to lock the entire collection whenever any write/update operation is happening, but we'll try it this way).

I also found code in views.files that belongs in tasks.files - gonna fix that in the process.

[gjost]

Slight correction: all locking is now done at the collection level. Previously it was a mix; some files mistakenly locked at the collection level, and even some entity tasks did. Since we record every file operation in git commits, and since git commits are repository-wide, I think that doing all locking at the collection level is probably the best option.

[gjost]

Fixed in ddr-local commit ed098f6 for package ddrlocal-develop_2.8.4b.

[pkikawa]

resolved in 284b dev

[pkikawa]

behavior still present in ddrlocal-ma-285

behavior from creating a new testing collection

(ddrlocal) ddr@DDREditor:/media/qnfs/kinkura/gold/ddr-testing-40231$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory)

    modified:   collection.json

Untracked files: (use "git add ..." to include in what will be committed)

    lock

no changes added to commit (use "git add" and/or "git commit -a")

[gjost]

Should be fixed in package ddrlocal-master_2.8.6.

[pkikawa]

Working in 286MA