Is a categorical X-axis possible in dynamic a scatter plot?

[carlocolantuoni]

for example, anything like the X-axis groupings in these:

[jorvis]

It's certainly possible, yes. Could you give an example dataset we have loaded which would fit to do something like this and what you would want the parameters to be?

[carlocolantuoni]

in general i just need to be able to plot using an X value, but label using a categorical value.

the " HuttCtxDevoLMDhs" and " Embryonic stem cell neural differentiation scRNAseq (Yao et al. 2017 Cell Stem Cell) " datasets could be examples where id like to plot age or time (respectively for the 2 data sets) on the X axis, but as categorical not numeric values. for example, with the yao data, there are 4 time points with many cells from each time. we could use a jittered vertical strip chart / scatter plot where y values are expression and x values are categorical time values, i.e. 0, 10, 24, and 54 days spaced as 4 equally spaced intervals. similarly, the data in " HuttCtxDevoLMDhs" have 4 ages (that are evenly spaced in time in this case, but in other data sets they wont be). id like to plot the expression on the y and age/class on the X. for most use cases this will be a number of non uniformly spaced time points that id like to plot across the X at regular intervals despite the fact that the time intervals are not equal.

it would be very similar to a violin plot where each violin happens to be (categorical) age. i would just want to select a column to use as the categorical value on the X. perhaps we would need to jitter samples around categorical X values if there are a lot of samples, so there is no overplotting.

that - make any sens?

On Wed, Apr 1, 2020 at 10:31 AM Joshua Orvis notifications@github.com wrote:

It's certainly possible, yes. Could you give an example dataset we have loaded which would fit to do something like this and what you would want the parameters to be?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#issuecomment-607284500, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7RZO6VKJ4RZKVERLJDRKNF3JANCNFSM4LVODGPQ .

-- Carlo

[carlocolantuoni]

Any of those examples make sense to you @jorvis?

[adkinsrs]

@carlocolantuoni I was playing around with a random dataset in gEAR and it does seem you can make a scatterplot with a categorical x-axis (see below) but the datapoints are in a straight line instead of jittered about. Is this the only change you would like?

[carlocolantuoni]

@adkinsrs thats cool - that is one of many things that would be nice here - do you have time this coming week to interact directly as we look at this and other plotly issues?

hows tuesday? noon work?

i will mark the other related issues with the plotly tag to get us going.

carlo

[adkinsrs]

@carlocolantuoni Tuesday at Noon should be fine!

This may be more of a discussion between @jorvis and myself, but if various feature additions and bug fixes introduce too much complexity into the plotting API modules, then it may be worth refactoring the code.

[carlocolantuoni]

Ok, we can figure how how much retooling will be necessary wen we talk tue - lookin forward to it, Carlo

[carlocolantuoni]

the x axis jitter could be an additional element in the display parameters within the view curation

[carlocolantuoni]

this extra display parameter should different if the selected x element is categorical or numeric

[adkinsrs]

It seems that Plotly's API only allows for jittering with datapoints that are supplementary to box or violin plots. May have to get down and dirty to implement a version of my own.

[hertzron]

Hi Shaun, is the problem with plotly graphs not filling the box figured out?

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Shaun Adkins notifications@github.com Sent: Tuesday, July 28, 2020 1:24:18 PM To: nemoarchive/analytics analytics@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [nemoarchive/analytics] Is a categorical X-axis possible in dynamic a scatter plot? (#109)

It seems that Plotly's API only allows for jittering with datapoints that are supplementary to box or violin plots. May have to get down and dirty to implement a version of my own.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/nemoarchive/analytics/issues/109#issuecomment-665161820, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEFF5X6H4P7YLVZV3BKSF53R54ATNANCNFSM4LVODGPQ.

[jorvis]

@hertzron - please try to keep questions related to the ticket they're in, else things get difficult to track. Created ticket for the space/padding issue: https://github.com/jorvis/gEAR/issues/777

[carlocolantuoni]

ok, a 1st fix might not be too bad here: for the " NHPforeBrainDevoAllenLMD.CtxDevo " dataset i have in NeMO (and several others), i have done the X value jittering already. if we could allow the addition of custom tick labels at specified X values, that would do the trick. for other more general applications we will want to be able to add jitter to the X values that a user has uploaded without jitter, but that will be more time consuming. can we try to add custom tick labels 1st? we would need to get a series of X axis positions and a series of labels of the same length from the user. carlo

On Tue, Jul 28, 2020 at 1:23 PM Shaun Adkins notifications@github.com wrote:

It seems that Plotly's API only allows for jittering with datapoints that are supplementary to box or violin plots. May have to get down and dirty to implement a version of my own.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#issuecomment-665161820, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7RE7OQ3EOFO4FBMGHLR54ATLANCNFSM4LVODGPQ .

-- Carlo

[adkinsrs]

@carlocolantuoni I have implemented a jitter slider in devel.umgear.org and it goes from 0 to 1 in 0.25 step increments. This slider will only show if the plot is a scatter plot and will only take effect with a categorical x-axis. I guess I could try to hide it if the x-axis is numerical but the gEAR UI code I have been working on does not distinguish between categorical and numerical groups.

Jitter of 0 Jitter of 0.25 Jitter of 0.5 **Jitter of 1

[carlocolantuoni]

looks great!

On Fri, Aug 7, 2020 at 3:12 PM Shaun Adkins notifications@github.com wrote:

@carlocolantuoni https://github.com/carlocolantuoni I have implemented a jitter slider in devel.umgear.org and it goes from 0 to 1 in 0.25 step increments. This slider will only show if the plot is a scatter plot and will only take effect with a categorical x-axis. I guess I could try to hide it if the x-axis is numerical but the gEAR UI code I have been working on does not distinguish between categorical and numerical groups.

Jitter of 0 [image: newplot] https://user-images.githubusercontent.com/5665914/89679940-09ebf300-d8c0-11ea-8136-f53bd73f0c21.png Jitter of 0.25 [image: newplot (1)] https://user-images.githubusercontent.com/5665914/89679939-09ebf300-d8c0-11ea-8723-a79a5a02deaf.png Jitter of 0.5 [image: newplot (2)] https://user-images.githubusercontent.com/5665914/89679938-09ebf300-d8c0-11ea-931a-9ebd7e0d13df.png **Jitter of 1 [image: newplot (3)] https://user-images.githubusercontent.com/5665914/89679937-09535c80-d8c0-11ea-8659-f1f308b57bbe.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#issuecomment-670670360, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7WGTYAZIEMAVVD3KCLR7RG3NANCNFSM4LVODGPQ .

-- Carlo

[carlocolantuoni]

i dont have any data to test this on in gear - all my data are in nemo. i cant access one form the other - correct? if it is super easy, can u do this in devel.nemo? if that would take any time/effort at all, i will just wait til its pushed to production in NeMO. thanks shaun, carlo

On Fri, Aug 7, 2020 at 3:35 PM Shaun Adkins notifications@github.com wrote:

Closed #109 https://github.com/nemoarchive/analytics/issues/109.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#event-3634512347, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7XTNSPQYAQYHJAGXBDR7RJONANCNFSM4LVODGPQ .

-- Carlo

[hertzron]

Let’s do on both

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Carlo Colantuoni notifications@github.com Sent: Friday, August 7, 2020 3:36:28 PM To: nemoarchive/analytics analytics@noreply.github.com Cc: hertzron hertzron@gmail.com; Mention mention@noreply.github.com Subject: Re: [nemoarchive/analytics] Is a categorical X-axis possible in dynamic a scatter plot? (#109)

i dont have any data to test this on in gear - all my data are in nemo. i cant access one form the other - correct? if it is super easy, can u do this in devel.nemo? if that would take any time/effort at all, i will just wait til its pushed to production in NeMO. thanks shaun, carlo

On Fri, Aug 7, 2020 at 3:35 PM Shaun Adkins notifications@github.com wrote:

Closed #109 https://github.com/nemoarchive/analytics/issues/109.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#event-3634512347, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7XTNSPQYAQYHJAGXBDR7RJONANCNFSM4LVODGPQ .

-- Carlo

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/nemoarchive/analytics/issues/109#issuecomment-670679796, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEFF5XZAMWIUNYRNGRMIR7DR7RJTZANCNFSM4LVODGPQ.

[adkinsrs]

I added the code to nemo-devel. Since there is no valid URL for that yet (due to some DNS resolution issues that are currently being resolved) you can navigate to https://34.73.68.111/ . Note that this URL is for this particular server image, and most likely will not be permanent for the future.

[carlocolantuoni]

Awesome, thanks! ive been using https://devel.nemoanalytics.org/ isnt that the same?

On Fri, Aug 7, 2020 at 4:29 PM Shaun Adkins notifications@github.com wrote:

I added the code to nemo-devel. Since there is no valid URL for that yet (due to some DNS resolution issues that are currently being resolved) you can navigate to https://34.73.68.111/ . Note that this URL is for this particular server image, and most likely will not be permanent for the future.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#issuecomment-670700370, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7QFJDUN6T22SO6YBR3R7RP4DANCNFSM4LVODGPQ .

-- Carlo

[carlocolantuoni]

when a categorical variable is used for the X-axis in the dynamic scatter plot the categories are ordered along the X-axis in an automatic fashion, but these cant be re-ordered like the legend/coloring can be reordered (at lest i cant how).

is there a way to make the X-axis category ordering custom?

heres a screen shot of what i am doing (that uses your awesome new jittering and point size features as well!). as you can see, the postnatal samples are to the left of the fetal samples due to the automatic text ordering of the categorical X-axis (numbers before letters i guess). id like to make the order follow age:

[carlocolantuoni]

and your new axis labels are on there too! nice.

[carlocolantuoni]

also noticing that in the screen shot i just added to the previor comment, there are mean and standard dev erorr bars overlayed. they are hardly visible, but i think it might be good to make them optional

[carlocolantuoni]

sorry - one more thing here, after i save the view and then go try to look at it, the same view i had screen capp'ed above looks like this - with loss of the jitter - thinking this may be linked to the refresh/default/save issue we already have under investigation in ticket #122 :

[adkinsrs]

That save-and-restore stuff will be on the docket to fix next week. I can also add the x- and y- axis ordering as options though I'm curious why they weren't options to begin with. There is also currently a bug with the ordering menu right now (see https://github.com/jorvis/gEAR/issues/784) that I plan to investigate as well.

The reason you see error bars is because I actually made a box plot, which allows you to add datapoints and set the jitter to them (cannot do with scatter plots). I just set the box plot's color equal to the plot background color in an attempt to hide it, and I remove hoverpoint references. Plotly actually has a thing called the "strip plot" which is exactly what we want, but it's not available in the API we use.

[adkinsrs]

@carlocolantuoni I learned I had to add the marker_size and jitter stuff to another file for the type of display you linked. Can you try again?

[carlocolantuoni]

great! will do 2nite, thanks!

[carlocolantuoni]

the save works in that it saves the larger point snow. but anytime i hit preview chart, it resets all the custom colors, and i cant get the jitter slider to do anything now.

On Mon, Aug 10, 2020 at 5:03 PM Shaun Adkins notifications@github.com wrote:

@carlocolantuoni https://github.com/carlocolantuoni I learned I had to add the marker_size and jitter stuff to another file for the type of display you linked. Can you try again?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#issuecomment-671588237, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7Q6ZLNBBXOSXP7S2WDSABOBHANCNFSM4LVODGPQ .

-- Carlo

[carlocolantuoni]

also - i'm doing all this on devel.nemoanalytics.org - i get this error when i try to go to devel.umgear.org

[image: image.png]

On Mon, Aug 10, 2020 at 8:42 PM Carlo Colantuoni colantuonicarlo@gmail.com wrote:

the save works in that it saves the larger point snow. but anytime i hit preview chart, it resets all the custom colors, and i cant get the jitter slider to do anything now.

On Mon, Aug 10, 2020 at 5:03 PM Shaun Adkins notifications@github.com wrote:

@carlocolantuoni https://github.com/carlocolantuoni I learned I had to add the marker_size and jitter stuff to another file for the type of display you linked. Can you try again?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#issuecomment-671588237, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7Q6ZLNBBXOSXP7S2WDSABOBHANCNFSM4LVODGPQ .

-- Carlo

-- Carlo

[adkinsrs]

When I uploaded to nemo-devel, I forgot to reload the apache service so that my explain what you saw there. I did not upload to gear-devel (and your image for the error did not appear on your comment)

[carlocolantuoni]

2 notes here:

1] i can now order the categories on the X axis (which is great!), but when i hit preview, they revert. then if i alter anything below the preview button, they re-revert again.

2] i can use the jitter successfully on 1 data set where the X is categorical, but not on another where X is numeric. see 2 screen caps below:

[adkinsrs]

I still need to make a few modifications where anything below the "Preview" button is reverted back to the original state. When this UI was originally written, the original intention was that when categories are reordered or colors are picked, the plot is automatically redrawn, but the underlying status of the orders/colors is not passed to the Preview button, so the plot uses default colors/orders. (Also I had not changed anything to allow ordering of the X-axis so I'm unsure how you are ordering that :-) )

I explicitly set jitter to only work in cases where the x-axis is categorical, since jitter in a numerical-numerical scatter plot didn't make sense to me (by offsetting the true coordinates of the plot). The AgeGW x-axis should be represented as categorical instead of numerical in my opinion, so that a) the jitter would make more sense regarding application and b) the half-year tick step value would disappear.

[carlocolantuoni]

all sounds good - thanks!

On Thu, Aug 13, 2020, 08:29 Shaun Adkins notifications@github.com wrote:

I still need to make a few modifications where anything below the "Preview" button is reverted back to the original state. When this UI was originally written, the original intention was that when categories are reordered or colors are picked, the plot is automatically redrawn, but the underlying status of the orders/colors is not passed to the Preview button, so the plot uses default colors/orders. (Also I had not changed anything to allow ordering of the X-axis so I'm unsure how you are ordering that :-) )

I explicitly set jitter to only work in cases where the x-axis is categorical, since jitter in a numerical-numerical scatter plot didn't make sense to me (by offsetting the true coordinates of the plot). The AgeGW x-axis should be represented as categorical instead of numerical in my opinion, so that a) the jitter would make more sense regarding application and b) the half-year tick step value would disappear.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#issuecomment-673448509, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7QIL23QKNX7XJV7TD3SAPMBLANCNFSM4LVODGPQ .

[adkinsrs]

Made a fix where the colors are updated when a new color category is chosen, and also new order lists are populated upon updates as well. This is in gear-devel

I also rewrote the plotting package to use the Plotly Express higher-level API, which has an explicit "strip plot" for adding jitter. However it only allows for jitter to be turned on or off. So I modified the jitter slider to be a jitter on/off checkbox.

Closing this ticket for now... if any new issues pop up, please add them to a new ticket, as it will be easier to track

[carlocolantuoni]

Cool, thanks!

On Tue, Aug 25, 2020 at 8:57 AM Shaun Adkins notifications@github.com wrote:

Closed #109 https://github.com/nemoarchive/analytics/issues/109.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nemoarchive/analytics/issues/109#event-3689965330, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH7KC7QO5SPPJGHZYNOH2JTSCOYKPANCNFSM4LVODGPQ .

-- Carlo