The public data cannot be downloaded (via download project/group/...) -- MoDa asks for user to login

[subirranjan1]

@omimo Takes can be downloaded now without being logged in. Please check and confirm.However while testing I came across that downloading a whole project at once doesn't work. Is it necessary?

@michaelnixon What is your idea about downloading a whole project?I understand download size for certain projects will be very big. Do you think that causes time out issue when trying to download the whole project "Affective motion Graph"

Also can you throw some detail about how assets table gets populated while importing the data for a project. I think I am missing few key points here.I know that asset table is being used to display filetype and also for export. As it has no records related to Affective Motion Graph Data which I imported, export function is not working for this project.

I am also planning to display the download size by summing the file sizes on each hierarchy.Any comments on that is appreciated.

[michaelnixon]

@subirranjan1 re: Downloading whole project Each controller has an export method to allow downloading. It travels down the hierarchy building up the appropriate file structure, then zips it all. When I left it, it probably needed to have some code added that would build up a text file to accompany the files, that would provide metadata for the files (i.e. the info that accompanies it in the hierarchy.) However, I thought I had tested it and it was functional at each level. You may be running into speed problems in constructing the zip files on the fly though, however, I don't really see a way around that.

re: assets table We are using the Paperclip plugin to handle attachments. The assets table technically handles any file uploaded into the system (whether it's a data_track or attachment of any other kind). That explains line 2 of the model: "belongs_to :attachable, polymorphic: true"

So, the assets table is where the more technical information about the file (i.e. that which the system generates, not the data provided by the researchers). Whenever a file is put into the system for any reason, it is recorded in the assets table.

[subirranjan1]

@michaelnixon So by that you mean while re importing the data by using rake batch import should also populate the asset table?But asset table was not populated when I re imported. Is it because of the path of the files in the excel sheet might be wrong. Example : /Volumes/Macintosh_Drive_3_1TB/Database2/AffecGraph-DBImport/Annotations/8Walk/19_HVHA_8_walk_sean1_sean_Take_001_annotations.txt

Is the above path correct or do I need to add any folder prefix in that path(FYI, I can't access that volume )?

[michaelnixon]

@subirranjan1 Yes, every file in the system must have a corresponding row in the assets table. If it's not being created, then there is a failure somewhere along the process of uploading.

Indeed, the first part of the import process is transferring the files to the server in a temporary folder. I've been using scp to move files to /media/Database_/data-for-import/. The path you're showing seems to be a local path on Phillipe's lab Mac, so it won't work at all.

Then, you need to make sure all the file paths go to the correct location, e.g.: /media/Database_/data-for-import/MovingStories-Reliability-25052014/mocap/bvh_rick_cleaned/104-DirectionB-Advacing-1.bvh

If you want, you could change the method so it would allow for downloads from the internet. For example, in another project, I changed the use of "File.open" to "URI.parse". However, this requires that all files are in a network-accessible location. Then, you "pay" the "cost" of downloading one at a time, and any errors in file transfer occur during the import process, which I believe is more complicated to trouble-shoot.

[subirranjan1]

Thanks @michaelnixon for the detailed explanation.

@omimo or @michaelnixon Can you provide over email Phillipe's lab mac details required to use scp to move files or the details to login the server wherever files sit now(and the rights to do scp). Thanks.

[michaelnixon]

I just tried to log in and the credentials I have don't seem to allow access anymore, so I think we'll need @omimo here.

[subirranjan1]

@michaelnixon Thanks. I will wait for @omimo to reply here.

[omimo]

I think this is done, right @subirranjan1?