HTTP Download appears to require Write access to the data object or collection

[simont]

I have a collection and a group which has READ access to that collection. From the command line, the group members can read files (iget, etc) but cannot upload (iput) files into that collection.

If I use GoRods to access the collection they can see the files but if I kick off an http download then this fails with a lack of permissions error. However, if I grant iRODS WRITE access to that collection then http now downloads work. This doesn't seem to be how this is meant to work as now my READ-only collection has to be R/W in order for users to Read (download) the files via the web.

The relevant parts of the http handler code are below, is GoRods writing any temp data anywhere during the ReadChunk() (NB writer is an http.ResponseWriter object, I don't imagine this is influenced by the iRODS write permissions in any way)

obj, err := con.DataObject(filePath)
            if err != nil {
                http.Redirect(writer, request, request.Referer(), http.StatusNotFound)
                return
            }

            attachmentString := "attachment; filename=\"" + file + "\""
            writer.Header().Set("Content-Disposition", attachmentString)
            writer.Header().Set("Pragma", "public")
            writer.Header().Set("Cache-Control", "must-revalidate, post-check=0, pre-check=0")
            writer.Header().Set("Content-Length", strconv.FormatInt(obj.Size, 10))

            obj.ReadChunk(4096, func(chunk []byte) {
                writer.Write(chunk)
            })

[jjacquay712]

Hey Simon,

Thanks for the report. Is there a specific error string you're seeing? I'm thinking it might have something to do with DataObj.LSeek() being called within the read functions, since that's a read/write helper function. You could surround your ReadChunk with a Printf to check for that:

fmt.Printf("%v \n", obj.ReadChunk(4096, func(chunk []byte) {
    writer.Write(chunk)
}))

I'll investigate this myself as well. Thanks!

[simont]

Here's what I get with that extra code:

2016-08-02 05:30:54.340437528 +0000 UTC: Fatal - iRods Open DataObject Failed: /tempZone/home/shared/reference_data/normal_kidney.jpg, rcDataObjOpen failed
2016-08-02 05:30:54.505819055 +0000 UTC: Fatal - iRods Open DataObject Failed: /tempZone/home/shared/reference_data/normal_kidney.jpg, rcDataObjOpen failed

I'm also seeing rodsLog entries like this, interestingly there are what appear to be two sets of messages here, just as there were two Fatal error messages above, not sure why that would be happening.

Aug  2 05:30:54 pid:3640 NOTICE: Agent process 29113 started for puser=tester and cuser=tester from 127.0.0.1
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29113 NOTICE: _rsGenQuery: genQuery status = -818000
Aug  2 05:30:54 pid:29113 NOTICE: rsGenQuery: _rsGenQuery failed, status = -818000
Aug  2 05:30:54 pid:29113 NOTICE: getDataObjInfo: rsGenQuery error, status = -818000
Aug  2 05:30:54 pid:29113 NOTICE: _rsGenQuery: genQuery status = -818000
Aug  2 05:30:54 pid:29113 NOTICE: rsGenQuery: _rsGenQuery failed, status = -818000
Aug  2 05:30:54 pid:29113 NOTICE: getDataObjInfo: rsGenQuery error, status = -818000
Aug  2 05:30:54 pid:29113 NOTICE: readAndProcClientMsg: received disconnect msg from client
Aug  2 05:30:54 pid:29113 NOTICE: Agent exiting with status = 0
Aug  2 05:30:54 pid:3640 NOTICE: Agent process 29113 exited with status 0
Aug  2 05:30:54 pid:3640 NOTICE: Agent process 29115 started for puser=tester and cuser=tester from 127.0.0.1
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsHome=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: created irodsCwd=/tempZone/home/rods
Aug  2 05:30:54 pid:29115 NOTICE: _rsGenQuery: genQuery status = -818000
Aug  2 05:30:54 pid:29115 NOTICE: rsGenQuery: _rsGenQuery failed, status = -818000
Aug  2 05:30:54 pid:29115 NOTICE: getDataObjInfo: rsGenQuery error, status = -818000
Aug  2 05:30:54 pid:29115 NOTICE: _rsGenQuery: genQuery status = -818000
Aug  2 05:30:54 pid:29115 NOTICE: rsGenQuery: _rsGenQuery failed, status = -818000
Aug  2 05:30:54 pid:29115 NOTICE: getDataObjInfo: rsGenQuery error, status = -818000
Aug  2 05:30:54 pid:29115 NOTICE: readAndProcClientMsg: received disconnect msg from client
Aug  2 05:30:54 pid:29115 NOTICE: Agent exiting with status = 0
Aug  2 05:30:55 pid:3640 NOTICE: Agent process 29115 exited with status 0

[simont]

Im seeing similar access permission issues when trying to get metadata for data objects that are Read only - I can't access the metadata unless I change the permissions to Write.

If I search for objects with the metadata then I can find the files, however, I am unable to get the metadata for those matching files when I display them on the web page.

This doesn't seem to the case for Collections, I can find sub-collections which are in these Read-only collections with metadata and still access/display their metadata without any problems.

[jjacquay712]

Thanks so much for digging into this issue. I've located the cause of the bug - all DataObjs are hardcoded to open as R/W.

To fix the problem, I plan on opening DataObjs as read-only by default. If any function requires a write, I'll close the handle and reopen as R/W. This will keep the existing GoRods syntax intact. I might also add a more explicit open functions, possibly OpenR() or OpenRW().

[simont]

Good news and bad news.

The good news - It looks like I can correctly download from the Read Only collections and I can still upload files so that seems to be working as expected.

The bad news - this change seems to have broken the Connection.QueryMeta(topic) function - or at least my code used to work before pulling the new update. I can't see any log messages anywhere in the rodsLog or my own logs, the code hits the QueryMeta() call and then hangs.

S.

[jjacquay712]

Oops, sorry about the QueryMeta() issue. That bug was a result of this commit that enables connection sharing between goroutines: https://github.com/jjacquay712/GoRods/commit/08a971d67d7b4c19f71267659d5c0b6cb5386083

Should work for you now, just pull down the latest commit from master.