Closed will-moore closed 4 years ago
@will-moore would it be possible to impose a file size limit? We often deal with CSVs in 10s of GBs which could potentially lead to pretty severe side effects.
@emilroz Done (see last commit). I've set the default to 100MB and this is configurable.
Created a 13 ROIs-strong csv using the Batch ROI export script on a Project.
See https://merge-ci.openmicroscopy.org/web/webclient/?show=project-15107 Login as user-3 But when the dropdown menu above cetral pane "Parade" is chosen, then in the top menu "...csv" is selected, I get an empty dialog with Error - Send feedback.
Actually, the problem is not that the whole app does not work. I can still use the lower menu, which adds columns, and it works fine.
the ticks are not there in FF - see above - the values in the graph are 8, 15 and 51
edit: This is a firefox-specific issue - works fine in Chrome
@pwalczysko You reported this already in 2018: https://github.com/ome/omero-parade/issues/43
Filtering for "Image" from the csv does not make sense and/or does not work:
image_id
and image_name
(the csv was created by Batch ROI export script)Image
P.S. After trying to put some numbers in (not text as previously), I can see that the filtering is happening, but still, no tooltip suggestions, and where is the id
from the name of the filter ? Image
is not sufficient to understand this filter.
As I have only one value of
t
in my csv, namely t=1, the histogram looks very funny (see above) and the slider is there - long and with the button, but it "does not work" - the UI in this case passes a wrong impression - maybe in such case (one value only) the histogram and the slider should not be there ?
The channel
column produces another illogical behaviour. Probably caused by my csv having in one case an image with 3 rows (3 channel image). The slider works, and it kind of makes sense, but only and solely because I know how the parade expects one row per image, how it takes randomly (I can see in this case rather that it takes the first value) the row for that image. If I do not know that, I would be again confused.
Area is another interesting case. As some of my images have ROIs which have no areas, they just vanish when this filter comes up. This has some justification. Nevertheless, it invites the question about the empty values in my csv, which is much more general. Does no value mean "filter me out of sight" ? Maybe better would be a warning+some other highlighting of the images (greying out ?) as the N/A
ones
The lenght filtering (2 valid values only) does not seem to make sense.
Even if I use the =
sign for filtering, I will not see my 2 images ever. Rounding errors ?
Couple of comments. Maybe some of them are generally about parade. I think https://github.com/ome/omero-parade/pull/75#issuecomment-588901886 is pertaining to this workflow though
I think #75 (comment) is pertaining to this workflow though
Discussed with @will-moore : the image_id
-> Image
mapping from the csv column header to the parade is a fixed feature of parade. Probably hard to improve that without starting a new re-design of parade.
Happy to merge then, as the PR still brings an important new functionality
Thanks @pwalczysko - Yes, I don't want to try and fix all the UI issues in this PR.
We should probably update the Batch_Image_Export.py
script to use Image
as a column name for image ID, as we do for tables.
Overall it looks fine but we should now start thinking of methods that we can re-use since we do the same over and over either in training example script/analysis examples or web app. It fixes a problem but does not allow re-usability for developers willing to use the API
I redeployed that last commit and added a CSV file to https://merge-ci.openmicroscopy.org/web/webclient/?show=project-6358 user-3 with an image
column.
This is working now for filtering, table data and plotting.
idr0021 does not have in the list of filter Batch_ROI_Export.csv
edit: discussed with @will-moore This is due to the fact that there is a limit set for large CSV. so the limit is working :-)
Yes, OMERO-web job had omero config set omero.web.parade.max_csv_size 1000000
and the Batch_ROI_Export.csv was over 1MB. Redeployed now without that setting and the csv is available.
The file is now available and working. Merging and tagging a new version of parade
This adds CSV support to OMERO.parade for filtering and table_columns / scatter plot.
To test:
Image
column orimage_id
column (NB: this can be created withBatch_ROI_Export.py
but may give confusing results if there is more than 1 row per Image ID..csv
file on the Project/Plate. NB: only alpha-numeric characters, underscore and.
are allowed in this list since parade UI uses the chosen option to the URL directly. Other characters are currently replaced by underscore. Alternative option is to relax the URL regex to allow more flexibility. Currently this isr'^filters/script/(?P<filter_name>[\w.]+)/$'
file.csv column
in the drop-down list. This allows choosing columns from multiple different CSV files.Batch_ROI_Export.py
to create a CSV since https://github.com/ome/omero-scripts/pull/156 adds Plate support.omero config set omero.web.parade.max_csv_size 1000000
. Any CSV file over this size should be ignored by filters and 'table data'. The default limit without this setting is 100MB.