kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.29k stars 616 forks source link

Duration and View Count on Video Posts #246

Closed ghost closed 3 years ago

ghost commented 3 years ago

Gathering duration and view count on video posts would be useful as well

neon-ninja commented 3 years ago

This does seem possible - this JSON is embedded in the HTML:

{
   "\u0040type":"VideoObject",
   "name":"Toon Meowscles - Fortnite | Facebook",
   "description":"Meowscles has been toon\u2019d up and has found himself in the Toon World! Available for a limited time in the Fortnite Shop.",
   "thumbnailUrl":"https:\/\/scontent.fakl1-3.fna.fbcdn.net\/v\/t15.5256-10\/fr\/cp0\/e15\/q65\/167895532_150957813581988_4114143851903042211_n.jpg?_nc_cat=108&ccb=1-3&_nc_sid=08861d&efg=eyJpIjoidCJ9&_nc_ohc=Vx5HaGddtlkAX_UQ58k&_nc_ht=scontent.fakl1-3.fna&tp=14&oh=ee3f3486a63d34401bd5bfa422d4f8f6&oe=60BCDBC6",
   "duration":"T26S",
   "uploadDate":"2021-05-07T00:03:44-07:00",
   "contentUrl":"https:\/\/video.fakl1-2.fna.fbcdn.net\/v\/t42.1790-2\/182266575_864408177447334_7249907038966977100_n.mp4?_nc_cat=109&ccb=1-3&_nc_sid=985c63&efg=eyJ2ZW5jb2RlX3RhZyI6InN2ZV9zZCJ9&_nc_ohc=wFKHE8vLHMoAX-E8M0H&_nc_ht=video.fakl1-2.fna&oh=acd61bf0e7cbdfb68875093c75a0501e&oe=609889F6",
   "publication":[
      {
         "\u0040type":"BroadcastEvent",
         "isLiveBroadcast":false
      }
   ],
   "\u0040id":"https:\/\/video.fakl1-2.fna.fbcdn.net\/v\/t42.1790-2\/182266575_864408177447334_7249907038966977100_n.mp4?_nc_cat=109&ccb=1-3&_nc_sid=985c63&efg=eyJ2ZW5jb2RlX3RhZyI6InN2ZV9zZCJ9&_nc_ohc=wFKHE8vLHMoAX-E8M0H&_nc_ht=video.fakl1-2.fna&oh=acd61bf0e7cbdfb68875093c75a0501e&oe=609889F6",
   "url":"https:\/\/video.fakl1-2.fna.fbcdn.net\/v\/t42.1790-2\/182266575_864408177447334_7249907038966977100_n.mp4?_nc_cat=109&ccb=1-3&_nc_sid=985c63&efg=eyJ2ZW5jb2RlX3RhZyI6InN2ZV9zZCJ9&_nc_ohc=wFKHE8vLHMoAX-E8M0H&_nc_ht=video.fakl1-2.fna&oh=acd61bf0e7cbdfb68875093c75a0501e&oe=609889F6",
   "datePublished":"2021-05-07T00:03:44-07:00",
   "interactionStatistic":[
      {
         "\u0040type":"InteractionCounter",
         "interactionType":{
            "\u0040type":"http:\/\/schema.org\/WatchAction"
         },
         "userInteractionCount":1845
      },
      {
         "\u0040type":"InteractionCounter",
         "interactionType":{
            "\u0040type":"http:\/\/schema.org\/ShareAction"
         },
         "userInteractionCount":0
      },
      {
         "\u0040type":"InteractionCounter",
         "interactionType":{
            "\u0040type":"http:\/\/schema.org\/LikeAction"
         },
         "userInteractionCount":7
      }
   ],
   "width":1920,
   "height":1080,
   "thumbnail":{
      "\u0040type":"ImageObject",
      "contentUrl":"https:\/\/scontent.fakl1-3.fna.fbcdn.net\/v\/t15.5256-10\/fr\/cp0\/e15\/q65\/167895532_150957813581988_4114143851903042211_n.jpg?_nc_cat=108&ccb=1-3&_nc_sid=08861d&efg=eyJpIjoidCJ9&_nc_ohc=Vx5HaGddtlkAX_UQ58k&_nc_ht=scontent.fakl1-3.fna&tp=14&oh=ee3f3486a63d34401bd5bfa422d4f8f6&oe=60BCDBC6",
      "width":1920,
      "height":1080
   },
   "headline":"Toon Meowscles - Fortnite | Facebook",
   "isFamilyFriendly":true,
   "genre":"Games, Puzzles, & Play",
   "keywords":"Games, Puzzles, & Play, Video Games, Action Video Games, Shooting Video Games, Survival Video Games, Battle Royale Video Games, Nintendo",
   "commentCount":1,
   "comment":[
      {
         "\u0040type":"Comment",
         "text":"I find this both scary and fascinating.",
         "interactionStatistic":{
            "\u0040type":"http:\/\/schema.org\/InteractionCounter",
            "interactionType":{
               "\u0040type":"http:\/\/schema.org\/LikeAction"
            },
            "userInteractionCount":0
         }
      }
   ],
   "inLanguage":"en",
   "contentSize":"172617.87kB",
   "videoQuality":"1080p",
   "author":{
      "\u0040type":"Organization",
      "name":"Nintendo",
      "url":"\/NintendoAUNZ\/"
   },
   "publisher":{
      "\u0040type":"Organization",
      "logo":{
         "\u0040type":"ImageObject",
         "url":"https:\/\/scontent.fakl1-3.fna.fbcdn.net\/v\/t1.6435-1\/fr\/cp0\/e15\/q65\/82211324_1534974216659706_3951746172929966080_n.jpg?_nc_cat=105&ccb=1-3&_nc_sid=c1fdac&efg=eyJpIjoidCJ9&_nc_ohc=g2_T-tMzLKwAX_mbJYs&tn=06Z0r1zk0VzMV1-x&_nc_ht=scontent.fakl1-3.fna&tp=14&oh=4adb1aac1cfe3404eef777a65e824974&oe=60BED9DA"
      },
      "name":"Nintendo",
      "url":"\/NintendoAUNZ\/"
   },
   "creator":{
      "\u0040type":"Organization",
      "image":"https:\/\/scontent.fakl1-3.fna.fbcdn.net\/v\/t1.6435-1\/fr\/cp0\/e15\/q65\/82211324_1534974216659706_3951746172929966080_n.jpg?_nc_cat=105&ccb=1-3&_nc_sid=c1fdac&efg=eyJpIjoidCJ9&_nc_ohc=g2_T-tMzLKwAX_mbJYs&tn=06Z0r1zk0VzMV1-x&_nc_ht=scontent.fakl1-3.fna&tp=14&oh=4adb1aac1cfe3404eef777a65e824974&oe=60BED9DA",
      "name":"Nintendo",
      "url":"\/NintendoAUNZ\/",
      "interactionStatistic":[
         {
            "\u0040type":"http:\/\/schema.org\/InteractionCounter",
            "interactionType":{
               "\u0040type":"http:\/\/schema.org\/FollowAction"
            },
            "userInteractionCount":5437412
         },
         {
            "\u0040type":"http:\/\/schema.org\/InteractionCounter",
            "interactionType":{
               "\u0040type":"http:\/\/schema.org\/LikeAction"
            },
            "userInteractionCount":5432793
         }
      ]
   },
   "about":"Meowscles has been toon\u2019d up and has found himself in the Toon World! Available for a limited time in the Fortnite Shop.",
   "transcript":"",
   "\u0040context":"https:\/\/schema.org"
}

Another example of a longer duration video: "duration":"T33M8S",

neon-ninja commented 3 years ago

This should do what you want - https://github.com/kevinzg/facebook-scraper/commit/d9ab00aa641ff861f7e5031b371677f0f1ccf503

ghost commented 3 years ago

Thank you

ghost commented 3 years ago

When will it be available from pip package?

neon-ninja commented 3 years ago

Once a few more people have tested it to make sure I haven't broken latest master with this commit ;)

neon-ninja commented 3 years ago

Published on pypi as 0.2.35

ghost commented 3 years ago

Thanks for the quick response.