ScriptSmith / instamancer

Scrape Instagram's API with Puppeteer
http://adamsm.com/instamancer
MIT License
398 stars 61 forks source link

[Question] 'Fake' Streaming #13

Closed Silentz0r closed 5 years ago

Silentz0r commented 5 years ago

Can i use this library to make a 'fake' stream to a user account?

Like fetching data on new post every X time and if it's a new post get all the photos/videos of the new post?

Thank you.

ScriptSmith commented 5 years ago

This isn't part of Instamancer's core functionality, but you can write something to do it yourself with Instamancer. There may be other instagram tools that do what you're after, but I'm not aware of any.

For a user this is difficult, because Instamancer doesn't retrieve all of a user's most recent posts unless it's in full mode, and posts are returned out of order in full mode. Also, it's kinda wasteful because it would require you to create and destroy a new browser instance each time you want to check for new posts.

Instead, I'd just do this in a loop every x seconds:

  1. Make an http request to the user's page on instagram with some http library
  2. Parse the data in the line that starts with: <script type="text/javascript">window._sharedData =
  3. Look for new post ids
  4. Use Instamancer.post(ids, options) to get the data for new posts
Silentz0r commented 5 years ago

Thank you for your idea, it's awesome. I'm trying to use your library but im getting this:

const Instamancer = require('instamancer');
async function getDataInstagrambyID() {
    let data = await instagramPhotos();
    console.log(Instamancer.post(data[0].node.id))
}
getDataInstagrambyID();

console: Object [AsyncGenerator] {}

I didn't install Instamancer globally because i don't need it. I just did npm i instamancer --save-dev

Thank you.

EDIT: As you can see, i'm recently learning about coding. I read a little further testing but i'm not sure what type of id should i use. shortcode or id? I learned that i have use .next(); in order to the function starts. But if i use id as '2099751656438958566' or shortcode as B0j0QGulQnm with id using headless: false i get /p/undefined if i use shortcode i get /p/B/ like it's using the first letter only. Could be i'm doing something wrong?

EDIT2: Using silent: false im getting in the console:

B  Total: 5  State: Scraping  Sleeping: 1  Scraped: 0 
0  Total: 5  State: Scraping  Sleeping: 1  Scraped: 0 
j  Total: 5  State: Scraping  Sleeping: 1  Scraped: 0 
...

Like doing letter by letter and not the whole shortcode. I've installed as -g aswell and using the command-line instamancer post [ids] it works, so im clueless, any help would be appreciate it.

npm -v: 6.9.0
node -v: 12.6.0
ScriptSmith commented 5 years ago

post(ids, options) takes an array of posts, your issue with it only accessing /p/B is because you're only passing it a single string.

Posts on Instagram have a field called id but it can't be used to retrieve posts, so we use the shortcode as the identifier.

Don't worry about global install, that's for the CLI tool.

Adapting your code, it'd look something like this:

const Instamancer = require('instamancer');
async function getDataInstagrambyID() {
    const data = await instagramPhotos();
    const posts = Instamancer.post(data.map(post => post.node.shortcode));
    for await (const post of posts) {
        console.log(post);
    }
}
getDataInstagrambyID();
Silentz0r commented 5 years ago

I'm closing this as resolved, sorry for wasting your time on this. It's working now with your code. Thank you very much, have a nice day. I need much to learn.

ScriptSmith commented 5 years ago

Great! Don't worry about wasting my time, I'm happy to help.

ScriptSmith commented 5 years ago

And if you're a new developer, you might find using something like Instaloader or Instalooter to be a bit easier to wrap your head around when you're writing this because there's less async/await involved

Silentz0r commented 5 years ago

Thank you, mind you one last question, if i want to use only 1 shortcode (the last one) how can i do that? I only need the recent one to compare if it's a new post.

nvm, new Array(data[0].node.shortcode) is the answer! Excellent! Thank you.