Possible issue of getting rate limited

[arteevraina]

There can be possibility that the device gets rate limited, when you make multiple requests on the same URL final res = await http.get(Uri.parse(Uri.encodeFull(url + "arteevraina" +"/?__a=1&__d=dis")));

I wanted to make sure that if anybody has been rate limited while getting the user data. Because if we get rate limited using this approach we might have to rebuild this feature again using some other alternative.

[viralvaghela]

Yes, Do you have any other ideas? We want to do it without authentication, and don't want to use official API and API Keys.

[arteevraina]

Yes, Do you have any other ideas? We want to do it without authentication, and don't want to use official API and API Keys.

We are also working on the solution to avoid using official Facebook/Instagram APIs. As of now, this is the only solution that we found out. But, in future there is potential of URL changing or getting rate limited.

[arteevraina]

Yes, Do you have any other ideas? We want to do it without authentication, and don't want to use official API and API Keys.

I forgot to mention that getting the HTML data of the Instagram Profile in the in app browser by running javascript inside the Flutter App should have been one of the possible solutions.

But, the HTML that we are getting by running javascript does not contain the data like followers, following and posts count that we actually need. But, when you change the URL from Instagram User Profile to Github profile or Tiktok Profile you actually get that data. This means Instagram does not want us to scrape that data.

[viralvaghela]

I have some ideas like we can make a web scraper and deploy it on any free server(alternative to Heroku) and make rest API, and in this package we can call the REST APIs and implement. but it will take time, am busy with some work and college, How's this idea? what do you think?

[arteevraina]

I have some ideas like we can make a web scraper and deploy it on any free server(alternative to Heroku) and make rest API, and in this package we can call the REST APIs and implement. but it will take time, am busy with some work and college, How's this idea? what do you think?

I think this way if you are getting HTML in the server and not on the frontend side would help you to get the data that you want basically. But, there is issue that your server may also get rate limited. Since that server will be making requests now for all the package users that consume the REST API.

It would have been safer if we do this HTML fetching on the frontend side because in that case device will make the request for only one user not for all the users that are using the package.

[viralvaghela]

Yes that's also true, can we bypass rate limit by adding random ussr agent in every API call from the package? Or adding session or something?

[gauravmehta13]

Any updates on this?