Bug with Date Format in plotly on LINUX from ggplot conversion to PLOTLY

[ahsanshah]

I moved my code from Windows to Linux R/Shiny server and I noticed that the same exact code that displays the proper DATE format in Windows shows as a different Date Time format on Linux. It seems that when I convert the ggplot to PLOTLY using ggplotly(), it behaves differently with the Date format than in Windows.

Here is the sample code:

p1 <- ggplot() + geom_smooth( method = gam, formula = y ~ s(x,bs="cc",k=12), data = test2 , aes(x = DT_STAY_DATE, y = test2[, input$yvar_p] , color = PROPERTY_NAME) ) + ylab(input$yvar_p)

ggplotly(p1)

I moved my code from Windows to Linux R/Shiny server and I noticed that the same exact code that displays the proper DATE format in Windows shows as a different Date Time format on Linux. It seems that when I convert the ggplot to PLOTLY using ggplotly(), it behaves differently with the Date format than in Windows.

Here is the sample code:

p1 <- ggplot() + geom_smooth( method = gam, formula = y ~ s(x,bs="cc",k=12), data = test2 , aes(x = DT_STAY_DATE, y = test2[, input$yvar_p] , color = PROPERTY_NAME) ) + ylab(input$yvar_p)

ggplotly(p1)

[ahsanshah]

Hi, thanks for the update on the Update. I am new to plotly so bear with me as I have 2 questions:

1. Does downloading from the developer tool via "devtools::install_github("ropensci/plotly")" ensure that I have the latest code case from github? Is there a direct pull from github that I need to do to get this latest update.
2. I did try doing the update via "devtools::install_github("ropensci/plotly")" to see if it resolves the above issue. I now see the SCROLLING LEGEND (which is great)...but the X Axis DATE TIME stamp issue still persists. Again, I do NOT see this issue on Windows. Why does GGPLOTLY lose my DATE values on the X axis. As you can see, because of this the 2 data sets are disjointed which is not what I want.

I also confirmed this is unrelated to my specific code. On LINUX, for some reason if I have a DATE X axis on a ggplot geom, PLOTLY does not retain that format when I do plot_ly or ggplotly(). However, this is not an issue on Windows.

Here is a simple example of the problem using the ECONOMICS data set.

t <- ggplot(economics, aes(date, pop)) t + geom_point()

ggplotly()

When I convert via GGPLOTLY notice the X AXIS is now showing DATETIME which I do not want.

[cpsievert]

devtools::install_github("ropensci/plotly") installs the master branch. You can install the dev branch that will fix this with devtools::install_github("ropensci/plotly@rewrite")

[ahsanshah]

Thanks you. I installed the DEV branch and the issue is indeed resolved now.

However, it seems that it also impacted my second data set from add_trace (diamonds representing events) disappear. Does the new code impact how I added a trace to my existing plotly. Here is my code...The X axis date is fine now but the add_trace points are gone.

Here is my code snippet of how I am adding a trace to my original geom_smooth:

output$vis_p <- renderPlotly({

p1 <- ggplot() + geom_smooth(
  method = gam,
  formula = y ~ s(x,bs="cc",k=12),
  data = test2 ,
  aes(x = DT_STAY_DATE, y = test2[, input$yvar_p] , color = PROPERTY_NAME)
) + ylab(input$yvar_p)

# p_gg<-plotly_build(p1) # Class PLOTLY
# p_gg$layout$showlegend = "TRUE"
p_gg<-ggplotly(p1)
add_trace(
  p_gg,
  x = startlocal,
  y = events.capacity,
  mode = "markers",
  marker = list(size=12,symbol=2),
  yaxis = "y2",
  opacity = .7,
  data = events_p(),
  text = paste0( "<b>",
                 nametext,
                 "</b><br>",
                 "Capacity : ",events.capacity,
                 "<br>",
                 "Duration : ",duration,
                 "<br>",
                 "Category : ",categories.name               
                 ),
  name = "Events",group = categories.name
) %>%
  layout(yaxis2 = list(side = "right", overlaying = "y"),
         showlegend = "TRUE",
         title="Events & Demand",
         width = 1000, height = 600, 
         margin = list(l=75,r=75,b=200,t=50,pad=5),
         legend=list(x=1, y=0, 
         font=list(family = "sans-serif",
         size = 9,
         color = "#000"))
  )

It seems that PLOTLY is ignoring my add_trace POINTS as well as Y AXIS on the left..but it does show the LEGEND.

Thoughts? I can raise a separate case for this if that works better.

[ahsanshah]

I cofirmed the new DEV build has an issue with ading a trace to an existing graph.

Here is a simple example which I found online which shows the same issue I am facing with code above.

p <- plot_ly(economics, x = date, y = uempmed, name = "unemployment") p %>% add_trace(y = fitted(loess(uempmed ~ as.numeric(date))))

In the previous MASTER Build:

In the new DEV build:

The above behavior is impacting my existing ggplot + add_trace as I cannot see the markers on the add_trace, Please advise.

[royr2]

Unlike before, you will need to specify all the aesthetics in each trace like so:

library(plotly)
p <- plot_ly(economics, x = date, y = uempmed, name = "unemployment")
p %>% add_trace(x = date, y = fitted(loess(uempmed ~ as.numeric(date))))

[ahsanshah]

I tried this suggestion for my example but it does not work as expected. Note that in my case..the GGPLOT converted to PLOTLY uses a completely different Data Set than the ADD_TRACE. Basically, I am overlaying 2 different data sets and using a second Axis Y2.

This worked fine with the previous MASTER build but does not with the DEV branch. Any idea what I am missing here:

Here is my code:

p1 <- ggplot() + geom_smooth( method = gam, formula = y ~ s(x,bs="cc",k=12), data = test2 , aes(x = DT_STAY_DATE, y = OCCUPANCY , color = PROPERTY_NAME))

p_gg<- ggplotly(p1) p_gg %>% add_trace( x = startlocal, y = events.capacity, mode = "markers", marker = list(size=12,symbol=2), yaxis = "y2", opacity = .7, data = ATLevents, text = paste0( "", nametext, "
", "Capacity : ",events.capacity, "
", "Duration : ",duration, "
", "Category : ",categories.name
), name = "Events",group = categories.name ) %>% layout(yaxis2 = list(side = "right", overlaying = "y"), showlegend = "TRUE", title="Events & Demand", width = 1000, height = 600, margin = list(l=75,r=75,b=200,t=50,pad=5), legend=list(x=1, y=0, font=list(family = "sans-serif", size = 9, color = "#000")) )

This is how it looks prior to the new DEV BUILD:

This is how it looks with the NEW DEV BUILD:

[ahsanshah]

Also...the Y and X axis seem to be overlapping with the values....this was not an issue with the prior MASTER build.

Let me know if this is a bug or I need to do something to correct this. Adding the aesthetic on ADD_TRACE does not seem to help in this case.

[ahsanshah]

Is there a way for me to just get the FIX for the LINUX DATE issue...versus the full DEV build? We cannot proceed with using Plotly if it cannot allow the overlay and dual legend/axis across the GEOM and the ADD_TRACE Markers as before. I hope the new build does not limit that functionality as that was one of the most important features of Plotly for us. Any insight is appreciated.

[cpsievert]

Can you provide test2 and ATLevents?

[ahsanshah]

I canb provide ATLevents but test2 data I cannot as it is internal system data which is secure. However, I can provide the str output for TEST2.

str(test2) 'data.frame': 1460 obs. of 14 variables: $ PROPERTY_NAME: chr "Hotel Atlanta GA" $ LATITUDE : num 33.8 33.8 33.8 33.8 33.8 ... $ LONGITUDE : num -84.4 -84.4 -84.4 -84.4 -84.4 ... $ CAT : chr "stay" "stay" "stay" "stay" ... $ STAY_DATE : chr "2015-01-01 00:00:00.0" "2015-01-01 00:00:00.0" "2015-01-02 00:00:00.0" "2015-01-02 00:00:00.0" ... $ DOWK_TYPE : chr "Weekday" "Weekday" "Weekend" "Weekend" ... $ P_STAY_DATE : POSIXct, format: "2014-12-31 19:00:00" "2014-12-31 19:00:00" "2015-01-01 19:00:00" "2015-01-01 19:00:00" ... $ DT_STAY_DATE : Date, format: "2015-01-01" "2015-01-01" "2015-01-02" "2015-01-02" ... $ YEAR : chr "2015" "2015" "2015" "2015" ... $ IS_LOYALTY : chr "N" "Y" "N" "Y" ... $ OCCUPANCY : num 45.1 10.1 90.6 26.7 91.9 ... $ AVG_LEAD_TIME: num 34.3 29.1 50.1 49 42.2 ... $ AVG_ROOM_RATE: num 119 132 121 116 116 ... $ REVENUE : num 55980 18887 129457 44156 131768 ...

All I am using is DT_STAY_DATE as the X axis on the GEOM.

I attached the ATLevents RDS.. ATLevents.zip

Here is the str for ATLevents:

str(ATLevents) 'data.frame': 36 obs. of 10 variables: $ events.category_id: chr "101" "101" "101" "101" ... $ events.id : chr "19026946108" "18346376506" "19358478731" "8893734399" ... $ nametext : chr "Premiumsteamers Carpet Care Carpet Cleaning of Metro Atlanta/Marietta, Ga." "iSpeakPR Supporters Drive" "BREAKFAST with The BOSS NATIONAL TOUR" "Join or Renew Your Empire Board Membership" ... $ startlocal : Date, format: "2015-10-10" "2015-08-27" ... $ endlocal : Date, format: "2016-10-10" "2017-08-27" ... $ events.capacity : int 12000 201100 787 800 1001 3000 3000 49 600 1000 ... $ events.locale : chr "en_US" "en_US" "en_US" "en_US" ... $ events.venue_id : chr "11717270" "11300966" "11933282" "4407575" ... $ categories.name : chr "Business & Professional" "Business & Professional" "Business & Professional" "Business & Professional" ... $ duration : num 367 732 146 366 422 282 245 884 181 368 ... Let me know if this helps or what else I can provide. Again, this works fine priod to the latest DEV codebase.

Thanks

[cpsievert]

If I can't reproduce your example, there's no guarantee I can't fix it.

This works for me, so I'm not sure why the second trace isn't showing up.

ggplotly(qplot(1:10, 1:10, color = rep(c("a", "b"), 5))) %>% 
    add_trace(x = 1:10, yaxis = "y2") %>% 
    layout(yaxis2 = list(side = "right", overlaying = "y"), legend = list(x=1, y=.5))

I'm pretty sure you'll have to move the legend position to the right for the second axis to appear

[ahsanshah]

Here is mock data for test2. I dont think you are trying what I am trying. You can execute the exact same code for test2 and ATLevents and you should be able to reproduce the issue.

Thanks

test2.zip

[ahsanshah]

Confirmed with the 2 data sets I sent that I can reproduce the issue:

[cpsievert]

Ah, the ggplot converter now converts dates/datetimes to the number of milliseconds since the UNIX epoch before sending to plotly.

Change this line:

x = startlocal,

to

as.numeric(startlocal) * 24 * 60 * 60 * 1000,

[ahsanshah]

Awesome....that worked and I see the markers. 2 follow up items:

1. I also see a milliseconds as I scroll on the plot for the X axis. Is there a way to override this behavior to show the actual dates. Would scale_x_date work?

1. I also see that my Legend placement is no longer working so I may have to play around with that. I explicitly positioned the legend to show at the bottom to not cover the Y2 Right side axis but seems that does not render as expected.

Any thoughts on these 2 items ?

Regardless, thanks for your assistance.