NaturalHistoryMuseum / scratchpads2

Scratchpads 2.0
http://scratchpads.org
GNU General Public License v2.0
199 stars 83 forks source link

Specimen media upload, annotation and presentation needs major improvements; I therefore decided to host media externally on DropBox #5239

Closed informatics-dev closed 5 years ago

informatics-dev commented 10 years ago

Description:

Last week I saw myself forced to decide that I give up on hosting my specimen images and movies on the macrostomorpha.info scratchpad for the moment, for a whole series of reasons (some of this may repeat stuff I have said previously, but it might be useful to have these points in one place). Instead I am currently hosting this material by including a link to a publically accessible DropBox folder in the Remarks field of every specimen, which reads:

Images and movies of this specimen are available in a public DropBox folder. Unfortunately hosting this data externally is necessary because the Scratchpads currently offer a very poor user experience for browsing images and movies.

For an example see http://macrostomorpha.info/node/3339

That is not a very kind assessment, but unfortunately perfectly true, at least for the kinds of extensive specimen data that I am using. Here are the main reasons: 1) given that I often have several dozens of images and quite a few movies for every specimen the upload and annotation process is close to useless. The dialog only shows me ten items at a time (sometimes I can select all ten and sometimes not), which would mean that I have to use some iterative process in uploading stuff, which would take essentially forever. 2) the image browser that presents the media is very slow, the images too small, and none of the many video formats that I have tried actually seem to be supported in a satisfactory way or at all (although I have tried many that you list as supported). I think the DropBox browser can serve as a good example of how such a browser should look (although it is also not perfect).

I don't like the DropBox solution very much, because it means that these specimen images will not be properly tagged with the species, specimens and sampling locations. But it is infinitely preferable over hosting the media on the scratchpad (an upside is that a user can easily download all images and movies and study them locally). I was close to giving up on the scratchpads entirely ten days ago, but using this solution I have been able to push the decision into the future. But the question is far from settled. Please give your utmost to consolidate functionality before you move on to implement new stuff.

Finally, I would appreciate a candid response the following question: "Do you actually intend to come up with a solution that would allow me to host my media on the scatchpad?"

In some sombre moments I have entertained the notion that you simply don't want to do this, because you are concerned about server load and/or storage space (the DropBox folder now has 6.5GB). If that is the case then please tell me and I'll try to find another solution. Given that I own the site URL, that might be less disruptive than one would think.

Cheers, Lukas

informatics-dev commented 10 years ago

Comment by Dimitrios Koureas

Dear Lukas,

Thank you for your comments and questions on Scratchpads. We do take into consideration all our users' feedback in order to improve the level of service provided.

Currently Scratchpads provide the functionality of adding infinite number of images to a specimen. This can be done either through the user interface or in bulk by populating and uploading the corresponding excel template files (Import > Nodes - Excel File Import > Specimen/Observation).

Images are displayed as thumbnails, but can be seen in large format when selected and also downloaded by any user in full resolution.

We do not apply any storage quota to any of the Scratchpads sites.

Scratchpads do provide the required functionality to host, organise and annotate media files using either the inbuilt interface or the import functionality. Documentation can be found at http://help.scratchpads.eu

Scratchpads were not built as a competitive solution to mass cloud storage sites. Scratchpads are a virtual research environment that enable individual researchers and research communities to collaborate, share knowledge, expose their work and publish online and in a standardised way. They facilitate the effective mobilisation, aggregation and re-use of biodiversity data. We conform to community standards in both our data structure model and data exchange models we use.

Once more thank you for your comments and useful feedback.

Kind regards, Dimitris

informatics-dev commented 10 years ago

Comment by lukas.scharer lukas.scharer

Dear Dimitrios,

Thanks you for your response to my email. In my view there is, and often has been, a striking discrepancy between what the scratchpads do (or rather what you developers want them to be able to do) and the reality in getting these functionalities to work for your users. I'll respond to your points inline below.

Dimitrios Koureas wrote:

Thank you for your comments and questions on Scratchpads. We do take into consideration all our users' feedback in order to improve the level of service provided.

I am glad to hear that, and I indeed have to give the developers credit for usually being responsive and patient.

Currently Scratchpads provide the functionality of adding infinite number of images to a specimen.

Agreed.

This can be done either through the user interface ...

Using this one will have to do it in increments of ten images at a time, or—because of the image selection bug I mentioned previously—often one image at a time). So with 50 images per specimen that is clearly not an option.

or in bulk by populating and uploading the corresponding excel template files (Import > Nodes - Excel File Import > Specimen/Observation).

I have previously commented on the duplication bug in the import function, and several users report issues today with similar problems. So presenting this as a solution is hard to swallow.

Moreover, how do I actually add images to a specimen? You say by populating an import excel sheet. The following is from the media(URL) column of import excel sheet of a specimen I just made in the sandbox, to which I have attached three files:

1|2|BTV 26.6.2014 21-07-16.jpg|public://BTV 26.6.2014 21-07-16.jpg|image/jpeg|831158|1|1403809706|image|83014486-5026-4a92-aa8a-248914b70b0d|Array|Array|Array|Array|Array|BTV 26.6.2014 21-07-16.jpg|1|2|2|BTV 26.6.2014 21-07-15.jpg|public://BTV 26.6.2014 21-07-15.jpg|image/jpeg|1065115|1|1403809706|image|9acb586d-4eda-40c9-b67b-a45fd9e91225|Array|Array|Array|Array|Array|BTV 26.6.2014 21-07-15.jpg|1|3|2|BTV 26.6.2014 21-07-12.jpg|public://BTV 26.6.2014 21-07-12.jpg|image/jpeg|1399329|1|1403809706|image|b26d1093-f6a1-4fb5-b462-c77dc646e746|Array|Array|Array|Array|Array|BTV 26.6.2014 21-07-12.jpg|1

What is this gibberish? Is this data structure documented anywhere? How am I supposed to generate this for 50 plus images per specimen (and dozens of specimens) if I don't have a clue what the data structure is?

Moreover, I have actually changed the file names via the import function (dropping the 'BTV ' prefix) after I added them to the specimen. The files are now listed in that way when I check the images, but for some reason this prefix still shows up here.

So given the issues with the import, I thought the slickgrid maybe the solution, But as I just posted yesterday, the slickgrid is also very buggy, so that is not an option either.

Images are displayed as thumbnails, but can be seen in large format when selected and also downloaded by any user in full resolution.

As I said, the viewer is not very effective. Identifying a picture that shows the structure of interested in is often not possible from the thumbnails, so that one is forced to look at many individual pictures in sequence, which is extremely slow and time consuming.

We do not apply any storage quota to any of the Scratchpads sites.

I am glad to hear that, because I have plenty of data I would like to see online.

Scratchpads do provide the required functionality to host, organise and annotate media files using either the inbuilt interface or the import functionality. Documentation can be found at http://help.scratchpads.eu

They intend to, but in effect don't. As I said, most of the video formats that you claim are supported do not work (or if they do, let the users know which exact codecs they should use). Moreover, the video viewer is even worse than the normal image browser. I often have videos focussing throught the animals, and to see things well I need to be able to look at videos frame, by frame. And by the way, the following quite is for historical context:

On 07.12.2007, at 13:13, Vincent Smith wrote:

2) The movies are an integral part of the descriptions, as they allow to >>visualize the structures in different focal planes. Would there be a way of >>incorporating these? I am aware that there is software to reduce such data to >>2D, but for internal structures I think this is not working so well.

We do not currently handle videos, but there is no reason why we could not deal >with videos in exactly the same way that we handle images. It would take a few >days for us to sort this out, but I'll add it to our list of things to do.

Seven years and counting.

Scratchpads were not built as a competitive solution to mass cloud storage sites. Scratchpads are a virtual research environment that enable individual researchers and research communities to collaborate, share knowledge, expose their work and publish online and in a standardised way. They facilitate the effective mobilisation, aggregation and re-use of biodiversity data. We conform to community standards in both our data structure model and data exchange models we use.

Given that I have been maintaining a scratchpad since December 2007, I am not sure what you are trying to tell me here.

Once more thank you for your comments and useful feedback.

You are welcome!

Cheers,

Lukas

informatics-dev commented 10 years ago

Comment by lukas.scharer lukas.scharer

I am reopening this issue, as I think there are many issues that I outlined here that have not been properly dealt with yet.

informatics-dev commented 9 years ago

Comment by lukas.scharer lukas.scharer

alycejenni commented 5 years ago

closing due to lack of activity for 4y - Lukas, if you read this and still feel these are relevant, please create a separate issue for each item.