dvingerh / PyInstaLive

Python script to download Instagram livestreams.
MIT License
547 stars 111 forks source link

Monitoring for Lives without getting blocked #124

Closed zylopr closed 3 years ago

zylopr commented 3 years ago

Hi there,

Just wondering if anyone has figured out a way to monitor for Lives without getting the 'suspicious activity' and subsequent 'change your password' prompts from Instagram?

If not, I've thought of a potential workaround, but I'm not sure how to implement it exactly. Basically, you have a spare Android phone (or PC running Bluestacks) with the Instagram app installed, then when a live notification comes in from the Instagram app, an app forwards that notification on to the PC where it'll trigger PyInstaLive to run. Anyone have any ideas how to make this work?

coder39248583 commented 3 years ago

@zylopr It doesn't even have to be a spare android phone. It can be your primary one, if you have one. On the android phone you can install "Macdroid" free version (alternatively, "Automate" free version also works, as does "Tasker," but that one costs money). Any of those apps can be configured to detect Instagram notifications and, in response, trigger custom HTTP get requests to a web server running on your PC -- passing along the notification text. Your server can parse the notification text to see if it is a live video, and who the user is, and make the appropriate call to PyInstaLive. This would of course require you to install a web server on your computer. There might be easier ways but they don't come immediately to mind. A good old fashioned apache/php setup would work. For Windows there are bundled softwares like XAMPP or MAMP that can set this up for you.

For example:

In the Macdroid app, on your android, you would create a new macro (consisting of a trigger and an action). Set the trigger to "Notification received" from Instagram App (you can optionally set a text filter in this step, but it might be easier for your web server to do it.)

For the action choose "HTTP GET" and set the URL to https://your.server.ip.address:port/whatever.php?title=[not_title]&text=[notification]&textlines=[not_text_lines]&tickertext=[not_ticker]

"not_title" is short for notification title, etc.. You might not need all those variables, but Instagram might change which ones serve the "Bob is starting a live video" or "Bob is going live now with Sarah" texts. Of course you would replace parts of the address to match your server address, but the variables should be typed just like they appear, with brackets.

Make sure to check the boxes "url encode parameters" and "http get (no web browser)". You can optionally have it check for an http response, but you don't have to worry about that if you don't want to.

Then, in your server's PHP page (or whatever other server language is most familiar to you) you would regex check the $_GET["text"], $_GET["textlines"], etc, variables, for a pattern matching the usual live notification, for example on a windows server you could do something like this:

$text = urldecode($_GET["text"]);

// e.g. "Zylopr: Bob started a live video"
// e.g. "Zylopr: Bob is going live with...."
if(preg_match('/\((.+)\): (.+) started a live video/U', $text, $matches) || 
   preg_match('/\((.+)\): (.+) is going live/U', $text, $matches))
{
  $yourAccountName = $matches[1]; // if you have multiple accounts
  $yourPassword = "yourpassforthataccount";
  $theirAccountName = $matches[2];

  // starts pyinstalive in a visible window on your PC and immediately
  // returns so we don't leave macdroid hanging
  pclose(popen("start \"customwindowtitle\" cmd /C pyinstalive -u $yourAccountName -p $yourPassword -d $theirAccountName" ,"r"));
}

This should get you started. Once you achieve basic functionality you can flesh out your program to do more, like deal with conflicting lives (I don't know if pyinstalive can run multiple instances, and even if it could it might look suspicious if you ran simultaneous downloads. You can create a list of users sorted by priority, and in the case of a conflict kill an existing download for a lower priority user. You can whip something up that sends a kill command like ctrl-c keystroke to any windows with the "customwindowtitle" you created, or something.

Let me know how it goes. Good luck.

zylopr commented 3 years ago

Thanks for that very detailed response, coder39248583. I had actually just managed to get it working earlier using a combination of Tasker, AutoRemote and EventGhost, which together, achieves the task without needing any coding. Tasker also lets me filter the notifications that get sent to the PC based on the notification's text.

PyInstaLive can indeed handle simultaneous downloads. From what I've seen, the actual downloading of the data doesn't need authentication (only to get the URL in the first place). The Python script that was posted in this forum recently that finds the segments of the Live prior to starting PyInstaLive doesn't need any user credentials at all to work, so it'd probably be OK to have several downloads going. Now the only thing that's missing is for someone to integrate that Python script into PyInstaLive.

Thanks again.

coder39248583 commented 3 years ago

@zylopr Glad you got it working! I took a look at the script to download previous (missed) live segments and it has the potential to make many thousands of bad requests to Instagram's servers, and miss portions of video, so I would keep that in mind. Individual live segments are numbered (18700.m4v, 18902.m4v, 19103.m4v...), usually incrementing 2001 or 2002 each segment. The script in question guesses the URLs of the missing segments, and when it finds one, it jumps ahead 2000 and increments its guesses by 1 until it finds the next. So in the best case scenario it would only make a few bad requests to Instagram's servers for each segment downloaded. The problem is that sometimes Instagram increments the segments slightly less than 2000, so jumping ahead 2000 might move you slightly past the correct segment, not only missing the download of that segment, but you are now incrementing by 1 and making ~2000 bad requests until the next find.

coder39248583 commented 3 years ago

@zylopr Also, I just discovered that pyinstalive often cuts recordings short by an indeterminate number of segments. (Some of?) those missed segments can be found in the dash manifest, the url of which is located in the the downloads json.

Given that this repository is inactive, relies on an inactive Instagram api, and misses available portions of video (both at the beginning and end of a live), it might be a good idea to fork it and revamp. I nominate you, sir.

zylopr commented 3 years ago

Thanks for the replies, @coder39248583.

Perhaps a solution to the script skipping a segment when it's just under a 2000 increment is to change it to something like "i += 1998"? It'd be a few more requests overall, but with a lower chance of missing a segment and also avoiding ~2000 bad requests when the increment is actually under 2000.

I think I have had instances where it's missed segments, because it sits there for a long period of time without any new segments being found. Despite this, it doesn't seem like it's caused any issues with Instagram blocking my IP address.

As for forking and revamping it, as much as I'd like to, unfortunately my programming skills amount to around about 0.

coder39248583 commented 3 years ago

@zylopr

The jumps for that given live were 1968, 1969, 2001, and 2002. What an awkward music collection. Jump sizes in other lives I checked ranged from 33 to 37367! Marty!!

So bad requests would be impossible to avoid using that method, to say the least.

However! I just realized something. These jump sizes refer to video segment lengths (in milliseconds). So checking the video length of the last segment downloaded will tell us how much to add to the number to receive the next video.

We know the first video is "...-init.m4v. I don't know if it has a length, or how to find the first URL with a number. I can do some experimenting. But this method has the potential of 0 bad requests and 0 missed segments.

I'll need the JSON or the included dash manifest URL of a recent live where the segment links are still active to investigate further.

zylopr commented 3 years ago

Here's a dash manifest URL: https://scontent.cdninstagram.com/hvideo-vll3/_nc_cat-100/v/rQBp4OuZBxyzlKzJc5oLn/_nc_ohc-U3FctwEyN7YAX_R1B-4/live-dash/dash-hd/17930152723586011.mpd?_nc_rl=AfBRlvciQ9Cy1GqoiqE&ccb=2-4&ms=m_CN&oh=52ccd49dc2aceee5a78f36d05fc7debf&oe=60C5A24B

Is that what you're after?

Someone else mentioned in another thread that when a guest joins/leaves a live stream, that's one thing that causes an irregular change in the segment numbers.

cykac04 commented 3 years ago

@zylopr Glad you got it working! I took a look at the script to download previous (missed) live segments and it has the potential to make many thousands of bad requests to Instagram's servers, and miss portions of video, so I would keep that in mind. Individual live segments are numbered (18700.m4v, 18902.m4v, 19103.m4v...), usually incrementing 2001 or 2002 each segment. The script in question guesses the URLs of the missing segments, and when it finds one, it jumps ahead 2000 and increments its guesses by 1 until it finds the next. So in the best case scenario it would only make a few bad requests to Instagram's servers for each segment downloaded. The problem is that sometimes Instagram increments the segments slightly less than 2000, so jumping ahead 2000 might move you slightly past the correct segment, not only missing the download of that segment, but you are now incrementing by 1 and making ~2000 bad requests until the next find.

Well I have been written another using aiohttp to find all segements by brute force. But it would be terrible if you have missed thousands of segements.

coder39248583 commented 3 years ago

@zylopr @cykac04 I have taken this discussion to the relevant issue: #121 Thanks for the manifest @zylopr