Veldrovive / MMM-Page-Selector

An easy way to set up and move between pages on a MagicMirror²
MIT License
35 stars 6 forks source link

Excessive API requests #27

Open roccodicarlo opened 4 years ago

roccodicarlo commented 4 years ago

@Veldrovive I first want to say your module works great.

I ahve created 7 Pages on my mirror, with each page displaying a different background themed collection from unsplash. I am using the Wallberry theme module.

In unsplash, I am limited to 50 API download requests per hour.

I noticed at start up, refresh, restart MM that each wallberry connection (I have 7 wallberry modules configured, each with a different collection theme) will make 6-8 calls (sometimes I have seen 10-12 API calls). The Wallberry module creator says that only 1 API call is made to download the image.

He believes your module may be triggering many refreshes or restarts which is causing the issue..

While testing my mirror project, I am constantly restarting the MM and this quickly uses up my unsplash quota...and so I have to wait for an hour for quota to reset.

Is your module restarting or refreshing many times at startup. By the way, after startup and after the first hour, everything goes to normal, and only 1 API call per page update occurs.

Veldrovive commented 4 years ago

I am not sure why it would be making so many requests. Page-Selector should only be refreshing the mirror one time. As a test, try supplying a position prop to every module. This will make it so that no refresh will occur. If the problem persists, then I will make a testbed and see why so many requests are being made.

roccodicarlo commented 4 years ago

I added a position statement to each module and I noticed a better behaviour but still it made about 3-4 API calls per module. I also notice that when I restart MM, it takes a long time (up to 30 min) before my default page is active. I notice that the config file is "Page-Selector" autogenerated version...and it doesn't load back the original config file until everything is working..I guess.

However if I look at the PM2 log, I notice that the MM is all loaded and should be ready quite quickly. The following is a look at the log, and you can see that the webpage is ready to access quite quickly. Since I am using a server mode, I need to wait for electron to bring up the web page, and I am assuming that page-selector is also waiting

pm2 TAIL LOG
[TAILING] Tailing last 15 lines for [mmstart] process (change the value with --lines option)
/home/pi/.pm2/logs/mmstart-out.log last 15 lines:
0|mmstart  | [2020-04-23 21:36:33.526] [LOG]    Connecting socket for: MMM-AlexaControl
0|mmstart  | [2020-04-23 21:36:33.536] [LOG]    MMM-AlexaControl helper, started...
0|mmstart  | [2020-04-23 21:36:33.548] [LOG]    Connecting socket for: WallberryTheme
0|mmstart  | [2020-04-23 21:36:33.563] [LOG]    Starting module helper: WallberryTheme
0|mmstart  | [2020-04-23 21:36:33.569] [LOG]    Connecting socket for: MMM-ip
0|mmstart  | [2020-04-23 21:36:33.580] [LOG]    Starting module helper: MMM-ip
0|mmstart  | [2020-04-23 21:36:33.593] [LOG]    Connecting socket for: MMM-LocalTransport
0|mmstart  | [2020-04-23 21:36:33.600] [LOG]    MMM-LocalTransport helper started ...
0|mmstart  | [2020-04-23 21:36:33.612] [LOG]    Connecting socket for: MMM-GooglePhotos
0|mmstart  | [2020-04-23 21:36:33.626] [LOG]    Connecting socket for: MMM-Todoist
0|mmstart  | [2020-04-23 21:36:33.634] [LOG]    Starting node helper for: MMM-Todoist
0|mmstart  | [2020-04-23 21:36:33.647] [LOG]    Connecting socket for: MMM-CalendarExt2
0|mmstart  | [2020-04-23 21:36:33.661] [LOG]    Sockets connected & modules started ...
0|mmstart  | [2020-04-23 21:36:33.664] [LOG]
0|mmstart  | Ready to go! Please point your browser to: http://localhost, 0.0.0.0:8080

/home/pi/.pm2/logs/mmstart-error.log last 15 lines:
0|mmstart  |    Before reporting problems, check http://wiki.x.org
0|mmstart  |    to make sure that you have the latest version.
0|mmstart  | Markers: (--) probed, (**) from config file, (==) default setting,
0|mmstart  |    (++) from command line, (!!) notice, (II) informational,
0|mmstart  |    (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
0|mmstart  | (==) Log file: "/var/log/Xorg.0.log", Time: Thu Apr 23 21:36:34 2020
0|mmstart  | (==) Using system config directory "/usr/share/X11/xorg.conf.d"
0|mmstart  | libEGL warning: DRI2: failed to authenticate
0|mmstart  | [4516:4827:0423/213723.257011:ERROR:object_proxy.cc(632)] Failed to call method: org.freedesktop.DBus.Properties.Get: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
0|mmstart  | [4516:4827:0423/213723.289142:ERROR:object_proxy.cc(632)] Failed to call method: org.freedesktop.UPower.GetDisplayDevice: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
0|mmstart  | [4516:4827:0423/213723.333258:ERROR:object_proxy.cc(632)] Failed to call method: org.freedesktop.UPower.EnumerateDevices: object_path= /org/freedesktop/UPower: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
0|mmstart  | [4747:1:0423/213740.793803:ERROR:child_process_sandbox_support_impl_linux.cc(81)] FontService unique font name matching request did not receive a response.
0|mmstart  | [4747:1:0423/213740.807559:ERROR:child_process_sandbox_support_impl_linux.cc(81)] FontService unique font name matching request did not receive a response.
0|mmstart  | [4747:1:0423/213807.741851:ERROR:child_process_sandbox_support_impl_linux.cc(81)] FontService unique font name matching request did not receive a response.
0|mmstart  | [4747:1:0423/213807.764641:ERROR:child_process_sandbox_support_impl_linux.cc(81)] FontService unique font name matching request did not receive a response.
Veldrovive commented 4 years ago

Huh, that definitely shouldn't happen. It shouldn't even make an autogenerated config. I should probably put in some more logging so that these things are easier to debug.

Is it possible for you to send your config file? I would be able to debug much more effectively with it in hand.

roccodicarlo commented 4 years ago

Here is the config file with all token and API keys XXX'd

/* Magic Mirror Config Sample
 *
 * By Michael Teeuw http://michaelteeuw.nl
 * MIT Licensed.
 *
 * For more information how you can configurate this file
 * See https://github.com/MichMich/MagicMirror#configuration
 *   ACZqyBrAxfet-yxk73JHkpP-bESRc2mJhrzQjM6SPKfAamTndrlt8MQSiPAn-Q7HzgtjRvxe1rn
 */

var config = {
    address: ["localhost", " 0.0.0.0"], // Address to listen on, can be:
                          // - "localhost", "127.0.0.1", "::1" to listen on loopback interface
                          // - another specific IPv4/6 to listen on a specific interface
                          // - "", "0.0.0.0", "::" to listen on any interface
                          // Default, when address config is left out, is "localhost"
    port: 8080,
    ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1","192.168.1.1/24", "::ffff:192.168.1.1/24","::1","192.168.10.1/24", "::ffff:192.168.10.1/24"], // Set [] to allow all IP addresses
    /*ipwhitelist: [],*/                                                       // or add a specific IPv4 of 192.168.1.5 :
                                                           // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.1.5"],
                                                           // or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
                                                           // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.3.0/28"],

    language: "en",
    timeFormat: 12,
    displaySeconds: "false",
    showPeriod: "false",
    units: "metric",
    modules: [
                {
            module: "MMM-Page-Selector",
            position: "top_right",
            config: {
                defaultPage: "Inspiration",
                displayTitle: false,
            /*  selectPageNotif: ["SELECT_PAGE"],*/
                incrementPageNotif: [],
                decrementPageNotif: [],
                                persistentPages:  true,
                            autoChange: {interval: 25920000}
                           }
        },
                {
        module: 'MMM-AlexaControl',
        name: "Alexa-Control",
       position: 'top_center', 
    pages: {all: "top_center"},
        config:{
            image: true,
            height: 60,
            width:60,
            pages: 7,
            pm2ProcessName: "mmstart",
            vcgencmd: "vcgencmd"
        }
    },
          // Base WallberryTheme adds new font, styles, and a rotating background image pulled from Unsplash.com
  {
    module: "WallberryTheme",
    name: "Inspirational",
    position: "fullscreen_below", // Required Position
    pages: {Inspiration: "fullscreen_below"},
    config: {
      unsplashAccessKey: "XXXXXXXXXXXXXXX", // REQUIRED
      collections: "9966805", // optional - leave empty for a random photo
      updateInterval:3600000,
  /*    orientation: "portrait",*/
      backgroundOpacity: 1,
      brightImageOpacity: 1
    }
  },
  {
    module: "WallberryTheme",
    name: "Baseball-theme",
    position: "fullscreen_below",  // Required Position
    pages: {Baseball: "fullscreen_below"},
    config: {
      unsplashAccessKey: "XXXXXXXXXXXXXXXXXXX", // REQUIRED
      collections: "9967998", // optional - leave empty for a random photo
      updateInterval:3600000,
  /*    orientation: "portrait",*/
      backgroundOpacity: .4,
      brightImageOpacity: .5
    }
  },
  {
    module: "WallberryTheme",
    name: "Toronto-theme",
    position: "fullscreen_below", // Required Position
    pages: {Toronto: "fullscreen_below"},
    config: {
      unsplashAccessKey: "XXXXXXXXXXXXXXXXXXXXXXXXX", // REQUIRED
      collections: "9967222", // optional - leave empty for a random photo
      updateInterval:3600000,
  /*    orientation: "portrait",*/
      backgroundOpacity: .4,
      brightImageOpacity: .5
    }
  },
  {
    module: "WallberryTheme",
    name: "Spring-theme",
    position: "fullscreen_below", // Required Position
    pages: {Spring: "fullscreen_below"},
    config: {
      unsplashAccessKey: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX", // REQUIRED
      collections: "9972492", // optional - leave empty for a random photo
      updateInterval:3600000,
  /*    orientation: "portrait",*/
      backgroundOpacity: .4,
      brightImageOpacity: .5
    }
  },
  {
    module: "WallberryTheme",
    name: "Waterfalls-theme",
    position: "fullscreen_below", // Required Position
    pages: {Waterfalls: "fullscreen_below"},
    config: {
      unsplashAccessKey: "XXXXXXXXXXXXXXXXXXXXXXXXXXX", // REQUIRED
      collections: "9977238", // optional - leave empty for a random photo
      updateInterval:3600000,
  /*    orientation: "portrait",*/
      backgroundOpacity: .4,
      brightImageOpacity: .5
    }
  },
  {
    module: "WallberryTheme",
    name: "Italy-theme",
    position: "fullscreen_below", // Required Position
    pages: {Italy: "fullscreen_below"},
    config: {
      unsplashAccessKey: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", // REQUIRED
      collections: "9977515", // optional - leave empty for a random photo
      updateInterval:3600000,
  /*    orientation: "portrait",*/
      backgroundOpacity: .4,
      brightImageOpacity: .5
    }
  },
  {
    module: "WallberryTheme",
    name: "TTC-theme",
    position: "fullscreen_below", // Required Position
    pages: {TTC: "fullscreen_below"},
    config: {
      unsplashAccessKey: "XXXXXXXXXXXXXXXXXXXXXXXXXX", // REQUIRED
      collections: "9977370", // optional - leave empty for a random photo
      updateInterval:3600000,
  /*    orientation: "portrait",*/
      backgroundOpacity: .4,
      brightImageOpacity: .5
    }
  },
  // WB-clock adds local time (Optional Module)
    {
    module: "WallberryTheme/WB-clock",
    name: "topright-clock",
   /* position: "top_right",*/ // highly suggest using top_bar position
    pages: {Baseball: "top_right",Toronto: "top_right",Spring: "top_right",Waterfalls: "top_right", Italy: "top_right", TTC:"top_right"},
    config: {
      localCityName: "Toronto", // optional
    /*  otherCities: [
        {name: "DC", timezone: "US/Eastern"}, // optional
        {name: "Anchorage", timezone: "US/Alaska"} // optional
      ] */
    }
  }, 
         {
    module: 'MMM-ip',
    name: "MMM-ipAddress",
   /* position: 'bottom_bar',*/
    pages: {all:"bottom_bar"},
    config: {
        showFamily: 'IPv4',
        showType: 'wlan0'
    }
},
               {
  module: 'MMM-LocalTransport',
  name: "TTC-High-Park",
  header: 'HIGH PARK TO OAKVILLE GO',
  /*position: 'bottom_center',*/
  pages: {Toronto: "bottom_center"},
  config: {
    api_key: 'XXXXXXXXXXXXXXXXXXXXXXXXXX',
    origin: 'High Park Subway Station, Toronto, ON',
    destination: 'Oakville Go Station, Oakville , ON',
    updatInterval: 9000,
    maximumEntries: 3,
    maxModuleWidth:380
  }
},
        {
            module: "alert",
        },
               {
  module: "MMM-GooglePhotos",
  name: "Large-Photos",
 /* position: "bottom_left",*/
  pages: {Baseball: "bottom_left",Spring: "bottom_left", Waterfalls: "bottom_left", Italy: "bottom_left", TTC: "bottom_left"},
  config: {
                albums: ["My Family", "Our Family"], // Set your album name. like ["My weddi$
                updateInterval: 1500 * 60, // minimum 10 seconds.
                sort: "random", // "old", "random","new"
                uploadAlbum: null, // Only album created by `create_uploadable_$
                condition: {
                        fromDate: null, // Or "2018-03", RFC ... format availab$
                        toDate: null, // Or "2019-12-25",
                        minWidth: null, // Or 400
                        maxWidth: null, // Or 8000
                        minHeight: null, // Or 400
                        maxHeight: null, // Or 8000
                        minWHRatio: null,
                        maxWHRatio: null,
                        // WHRatio = Width/Height ratio ( ==1 : Squared Photo, $
                },
        /*      autoInfoPosition: true,*/
                showWidth: 630, // These values will be used for quality of dow$
                showHeight: 500,
                timeFormat: "YYYY/MM/DD HH:mm", // Or `relative` can be used.
  }
},
{
  module: "MMM-GooglePhotos",
  name: "Small-Photos",
  /*position: "bottom_left",*/
  pages: {Toronto: "bottom_left"},
  config: {
                albums: ["My Family", "Our Family"], // Set your album name. like ["My weddi$
                updateInterval: 1500 * 60, // minimum 10 seconds.
                sort: "random", // "old", "random","new"
                uploadAlbum: null, // Only album created by `create_uploadable_$
                condition: {
                        fromDate: null, // Or "2018-03", RFC ... format availab$
                        toDate: null, // Or "2019-12-25",
                        minWidth: null, // Or 400
                        maxWidth: null, // Or 8000
                        minHeight: null, // Or 400
                        maxHeight: null, // Or 8000
                        minWHRatio: null,
                        maxWHRatio: null,
                        // WHRatio = Width/Height ratio ( ==1 : Squared Photo, $
                },
        /*      autoInfoPosition: true,*/
                showWidth: 310, // These values will be used for quality of dow$
                showHeight: 300,
                timeFormat: "YYYY/MM/DD HH:mm", // Or `relative` can be used.
  }
},

                {

                 module: 'MMM-Todoist',
                 name: "Shopping-List",

                 /*position: 'bottom_right',*/   // This can be any of the regions. Best results in left or right regions.
                 pages: {Baseball: "bottom_right",Toronto: "bottom_right",Spring: "bottom_right",Waterfalls: "bottom_right", Italy: "bottom_right", TTC:"bottom_right"},

                 header: 'Shopping List', // This is optional

                 config: { // See 'Configuration options' for more information.

                 accessToken: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', //Rocco's Token 
             /*    accessToken: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX', //Pellegri Token */

                         maximumEntries: 60,

                         updateInterval: 10000, // Update every 10 minutes

                         projects: [ 2222961728 ], //Rocco this entry is mandatory  
                      /*   projects: [ 2223375174 ], //Pellegrini*/
                         showProject: false,
                         maxTitleLength: 20,

                         fade: false

          }

        },
                {

                 module: 'MMM-Todoist',
                 name: "Todo-List",

                /* position: 'bottom_right',*/   // This can be any of the regions. Best resul$
                 pages: {Baseball: "bottom_right",Toronto: "bottom_right",Spring: "bottom_right",Waterfalls: "bottom_right", Italy: "bottom_right", TTC:"bottom_right"},

                 header: 'TO-DO LIST', // This is optional

                 config: { // See 'Configuration options' for more information.

                 accessToken: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXX', //Rocco Token 
            /*     accessToken: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXX', //Pellegrini Token */
                         maximumEntries: 60,

                         updateInterval: 10000, // Update every 10 minutes

                         projects: [ 2222961232, 2230112741 ], //Rocco this entry is mandatory
                      /*   projects: [ 2223375109 ], //Gloria*/

                         showProject: false,
                         maxTitleLength: 20,

                         fade: false

      }

        },

    /*  {
            module: "updatenotification",
            position: "bottom_bar",
        },*/

    /*  {
            module: "clock",
            position: "top_right"
        }, */
                {
                      module: 'MMM-CalendarExt2',
                         name: "Calendar",
                         pages: {Baseball: "top_right",Toronto: "top_right",Spring: "top_right", Waterfalls: "top_right", Italy: "top_right", TTC: "top_right"},
                        config:  {
                         calendars: [
                          {
                              url: "https://calendar.google.com/calendar/ical/XXXXXXXXXXXXX%40gmail.com/XXXXXXXXXXXXXXXXXXX/basic.ics",
                      /*      url: "https://calendar.google.com/calendar/ical/XXXXXXXXXXXX1000%40gmail.com/XXXXXXXXXXXXXXXXXXXXXXX/basic.ics", */
                            name: "Monthly Appointments",
                            scanInterval: 90000,
                         },
              /*           {
                           url: "https://www.calendarlabs.com/templates/ical/Canada_Holidays.ics",
                           name: "Holidays",
                           icon: "twemoji-flag-for-flag-canada",
                         },  */
            /*             {
                          url: "https://www.calendarlabs.com/ical-calendar/ics/76/US_Holidays.ics",
                          name: "Holidays",
                          icon: "twemoji-flag-for-flag-canada",
                          },*/
               /*          {
                          url: "https://calendar.google.com/calendar/embed?src=en.canadian%23holiday%40group.v.calendar.google.com",
                          name: "Holidays",
                          afterDays: 365,
                          scanInterval: 30000,
                         },*/
                         {
                         url: "http://www.google.com/calendar/ical/vfnitvbsa4j4cg866m0oc2tses%40group.calendar.google.com/public/basic.ics",
                         name: "Holidays",
                         },
                       ],
                       views: [
                         {
                         name: "Upcoming Events",
                         mode: "upcoming",
                         slotTitle: "UPCOMING EVENTS",
                         position: "top_right",
                        /* pages: {Baseball: "top_right",Spring: "top_right", Waterfalls: "top_right", Italy: "top_right", TTC: "top_right"},*/
                         hideOverflow: true,
                         slotMaxHeight: "340px",
                         calendars: [],
                         useEventTimeRelative: false,
                        },
                         {
                         name: "Family Calendar",
                         mode:  "week",
                         position: "middle_center",
                        /* pages: {Baseball: "middle_center",Spring: "middle_center", Waterfalls: "middle_center", Italy: "middle_center", TTC: "middle_center"},*/
                         slotCount: 4,
                         frontNow: 0,
                         SlotTitleFormat: "D.",
                         className: "monthClass",
                         slotMaxHeight: "100px",
                         calendars: [],
                       /*  position: "top_left",*/
                      },
/*                        {
                         Name: "Todays Events",
                         mode: "current",
                         slotTitle: "Todays Events",
                         useEventTimeRelative: false,
                         hideOverflow: true,
                         slotMaxHeight: "150px", 
                         calendars: [],
                         position: "top_bar",
                       },*/
                      ],
                       scenes: [
                         {
                          name: "DEFAULT",
                         },
                      ],
                  },
              },
                   {
                        module: "currentweather",
                        name: "Todays-Weather",
                       /* position: "top_left",*/
            pages: {Baseball: "top_left",Toronto:"top_left",Spring: "top_left", Waterfalls: "top_left", Italy: "top_left", TTC: "top_left"},
                        config: {
                                location: "Toronto,Canada",
                                locationID: "6167865",  //ID from http://bulk.openweatherm$
                                appid: "XXXXXXXXXXXXXXXXXX"
                        }
                },
                {
                        module: "weatherforecast",
                        name: "Forecast",
                      /*  position: "top_left",*/
            pages: {Baseball: "top_left",Toronto: "top_left",Spring: "top_left", Waterfalls: "top_left", Italy: "top_left", TTC: "top_left"},
                        header: "Weather Forecast",
                        config: {
                                location: "Toronto,Canada",
                                locationID: "6167865",  //ID from http://bulk.openweatherm$
                                appid: "XXXXXXXXXXXXXXXXXXXXXXXXXXX"
                        }
                },
    ]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== "undefined") {module.exports = config;}
roccodicarlo commented 4 years ago

I also sent you an email with the screen renderances for each page. I also wanted to mention that my internet has suddenly become slower and I am thinking some of these API's are failing or timing out...and this may cause resets

roccodicarlo commented 4 years ago

1 more thing, I noticed that the GooglePhotos module does not rotate/update photos when I have pages enabled. If I access new page (with the same googlePhotos enabled), it seems to update photo at the first access tp page, and then it stops updating.

roccodicarlo commented 4 years ago

Actually after a little bit more testing. I have found the following.

I have 2 instances of GooglePhotos module. 1 instance is for pages 2,4,5,6,7) the other instance is for Page 3.

The google photos only change when I switch between pages (2,4,5,67,) and page 3 or vice versa. switching between pages 2,4,5,6,7 does not update/change the photo.

And of course, once on a page, the Photo updating in the module stops working

roccodicarlo commented 4 years ago

Ok I determined that the update interval in GooglePhotos is only activated when the page is NOT being viewed or not the active page. And so after you switch back to the page with the GooglePhoto's and time/update interval had already elapsed before switching pages, then new photo is displayed.

I show the sequence below; - assuming GooglePhotos update interval is 2 minutes You are on Page 2 with the 1st instance of GooglePhoto's (large photo) No Update occurs for as long as you are on page you can switch to any of the pages (4,5,6,7) with this 1st instance of GooglePhotos and no update every occurs You switch to Page 3 with the 2nd instance of GooglePhoto's (small photo) you will get a new photo initially this new photo will never update you stay on this page until the 2 minute update interval has expired then you switch to any of the pages (2,4,5,6,7) and a new photo is displayed

I hope the above helps describe the Issue

roccodicarlo commented 4 years ago

I re-tested with only a single googlephotos instance (the large one on pages 2,4,5,6,7). It seems that the update on googlephotos worked, but after a few hours it stopped updating (everything else was working fine). Switching to other pages did not trigger an update. So, the state of googlephotos was exactly like the state as discussed before.. It seems that googlephotos update interval gets corrupted when using page-selector. Note, it works fine without page-selector.