Instafeed no longer pulling through

[tomhewitt2201]

Describe the bug:

Instagram feed isn't loading on multiple sites where we use this plugin. It's happening across multiple environments as well for each site including development server and local environment. Until recently they were all working without any issues.

Steps to reproduce:

1. NA - Load site and nothing is pulling through.

Where is it happening?

• [ X] On a server
• [X ] Local dev environment

What are you requesting: Username: ... Hashtag: ...

Config file:

<?php

return [
    // Replace with your personal key
    'proxyKey' => '********************',
    'useProxy' => true,
    'timeout' => 10,
];

Software versions: Plugin Version: 2.1.0 PHP Version: 8.2.13 CraftCMS Version: 4.5.13

[kringkaste]

Hi, thanks for contacting us. We are investigating this at the moment. Instagram seems to have changed the structure of their site. So we will come back to you asap.

[danfathom]

We're having the same issue. Do you guys have a rough estimate of when/if you could get this fixed?

thanks!

[deevanych]

+1. same problem

[brambeekman]

Same here :(

[danfathom]

Hi, do you have an update on if/when this will be fixed?

[laurabennett]

Me too!

[jamesmacwhite]

It certainly does seem like something related to the DOM structure of the Instagram public profile side.

The edges array in the JSON response through the proxy is empty. The request is technically working fine still, but the DOM parsing is likely no longer returning the required content, due to the changes.

[kringkaste]

We are testing a solution at the moment. We release a new version asap.

[jamesmacwhite]

Great news! Thanks for the update. I certainly appreciate the challenges of maintaining the DOM parsing side of things.

[kringkaste]

@tomhewitt2201 @laurabennett @jamesmacwhite @deevanych We just released a new version. Could you test this on your sites? It should work with and without proxy.

[jamesmacwhite]

@kringkaste Thank you for this. I am unfortunately on Craft CMS 3, will this be backported? I cannot test this directly in the project it is used in, but I can potentially spin up a DDEV Craft CMS 4 environment to confirm the plugin changes.

[kringkaste]

@jamesmacwhite Yes, we will backport this.

[laurabennett]

The site I'm working with is also on Craft 3.

On Thu, Jan 4, 2024 at 11:30 AM Tom Schulze @.***> wrote:

@jamesmacwhite https://github.com/jamesmacwhite Yes, we will backport this.

— Reply to this email directly, view it on GitHub https://github.com/codemonauts/craft-instagram-feed/issues/70#issuecomment-1877400625, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYJKJXTUU4JR5Q7SZDIIITYM3KKVAVCNFSM6AAAAABA6C7LD2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZXGQYDANRSGU . You are receiving this because you were mentioned.Message ID: @.***>

--

Laura Bennett digital diva

drive > P.O. Box 2838, 170 Kearsarge Street North Conway, NH 03860 603.356.3030 @.***

[jamesmacwhite]

@kringkaste I spun up a Craft CMS 4 environment with the proxy key for the Craft CMS 3 project and tested the PHP getFeed() function and it now returns data again!

Interestingly, my Postman API mockup of the proxy call to: https://igproxy.codemonauts.com/[accountname]/ seems to still be returning blank on the edges array, if that's something to flag, but otherwise through Twig it is working.

Edit: Potentially this seems to be something related to Cookies, as look at the diff, that's part of the changes. Edit 2: After clearing the caches, it appears the response is returning empty again. However disabling the proxy, it appears to work OK, so potentially an issue with the proxy? I've previously had to use the proxy as it's never worked without.

[tomhewitt2201]

Hi @kringkaste I've updated the plugin to the latest version about 24 hours ago and cleared my cache in the CMS but doesn't seem to be pulling anything through. Is there any additional set up or changes I need to make to my config to get this working?

[robzor]

Hi there, I've updated to the latest version and it works if i set useProxy to false in the config, thank you for sorting this out!

@kringkaste do you think I will need to go back to using the proxy in future?

[mark-chief]

Seems to be working on some sites, but one in particular I get this error. Proxy disabled, Craft 4.5.12

codemonauts\instagramfeed\services\InstagramService::extractMedia(): Argument #1 ($response) must be of type array, null given, called in /home/craft/vendor/codemonauts/craft-instagram-feed/src/services/InstagramService.php on line 128

[robzor]

@mark-chief I got that and then invalidated the template instagram feed and cache a second time and it seemed to go away. odd!

[mark-chief]

@robzor That doesn't seem to work for me, i nuked every cache just in case. Error goes if I re-enable the proxy, but then the feed stops.

[robzor]

@mark-chief sounds like they need to put an extra check in the plugin to catch that then, mine did go away, what do you get if you enable the debug option in the config file?

[robzor]

I've just deployed some more code to staging and the error is back, being triggered by this template code:

{% for item in craft.instagram.getFeed()|slice(0,6) %}

And the error stack trace:

TypeError: codemonauts\instagramfeed\services\InstagramService::extractMedia(): Argument #1 ($response) must be of type array, null given, called in /var/www/html/vendor/codemonauts/craft-instagram-feed/src/services/InstagramService.php on line 128 and defined in /var/www/html/vendor/codemonauts/craft-instagram-feed/src/services/InstagramService.php:279

Stack trace:
#0 /var/www/html/vendor/codemonauts/craft-instagram-feed/src/services/InstagramService.php(128): codemonauts\instagramfeed\services\InstagramService->extractMedia(NULL)
#1 /var/www/html/vendor/codemonauts/craft-instagram-feed/src/services/InstagramService.php(69): codemonauts\instagramfeed\services\InstagramService->getInstagramAccountData('chewvalleytrees')
#2 /var/www/html/vendor/codemonauts/craft-instagram-feed/src/variables/InstagramFeedVariable.php(11): codemonauts\instagramfeed\services\InstagramService->getFeed('chewvalleytrees')
#3 /var/www/html/vendor/twig/twig/src/Extension/CoreExtension.php(1607): codemonauts\instagramfeed\variables\InstagramFeedVariable->getFeed()
#4 /var/www/html/vendor/craftcms/cms/src/helpers/Template.php(129): twig_get_attribute(Object(craft\web\twig\Environment), Object(Twig\Source), Object(codemonauts\instagramfeed\variables\InstagramFeedVariable), 'getFeed', Array, 'method', false, false)
#5 /var/www/html/storage/runtime/compiled_templates/6d/6db21285aef7b5b16fe02a6f74605aacc9ea84b2e7b0036046c29dc304468a55.php(391): craft\helpers\Template::attribute(Object(craft\web\twig\Environment), Object(Twig\Source), Object(codemonauts\instagramfeed\variables\InstagramFeedVariable), 'getFeed', Array, 'method')
#6 /var/www/html/vendor/twig/twig/src/Template.php(171): __TwigTemplate_d41947c498b307fa4b7abd40e162d2ff89ad2e8b041de9884ee947be775338a4->block_main(Array, Array)
#7 /var/www/html/storage/runtime/compiled_templates/22/22282c7df56ec70caadf9e1c4329ffc194a53877a83d3b807c500e29ddbe0067.php(123): Twig\Template->displayBlock('main', Array, Array)
#8 /var/www/html/vendor/twig/twig/src/Template.php(394): __TwigTemplate_0c4a2c3bb0e3467032e537e9146ce4aa6359de72c3dd58a737a79dfa4ef89641->doDisplay(Array, Array)
#9 /var/www/html/vendor/twig/twig/src/Template.php(367): Twig\Template->displayWithErrorHandling(Array, Array)
#10 /var/www/html/storage/runtime/compiled_templates/6d/6db21285aef7b5b16fe02a6f74605aacc9ea84b2e7b0036046c29dc304468a55.php(60): Twig\Template->display(Array, Array)
#11 /var/www/html/vendor/twig/twig/src/Template.php(394): __TwigTemplate_d41947c498b307fa4b7abd40e162d2ff89ad2e8b041de9884ee947be775338a4->doDisplay(Array, Array)
#12 /var/www/html/vendor/twig/twig/src/Template.php(367): Twig\Template->displayWithErrorHandling(Array, Array)
#13 /var/www/html/vendor/twig/twig/src/Template.php(379): Twig\Template->display(Array)
#14 /var/www/html/vendor/twig/twig/src/TemplateWrapper.php(40): Twig\Template->render(Array, Array)
#15 /var/www/html/vendor/twig/twig/src/Environment.php(277): Twig\TemplateWrapper->render(Array)
#16 /var/www/html/vendor/craftcms/cms/src/web/View.php(482): Twig\Environment->render('index.twig', Array)
#17 /var/www/html/vendor/craftcms/cms/src/web/View.php(535): craft\web\View->renderTemplate('index.twig', Array)
#18 /var/www/html/vendor/craftcms/cms/src/web/TemplateResponseFormatter.php(57): craft\web\View->renderPageTemplate('index.twig', Array, 'site')
#19 /var/www/html/vendor/yiisoft/yii2/web/Response.php(1100): craft\web\TemplateResponseFormatter->format(Object(craft\web\Response))
#20 /var/www/html/vendor/craftcms/cms/src/web/Response.php(337): yii\web\Response->prepare()
#21 /var/www/html/vendor/yiisoft/yii2/web/Response.php(340): craft\web\Response->prepare()
#22 /var/www/html/vendor/yiisoft/yii2/base/Application.php(390): yii\web\Response->send()
#23 /var/www/html/web/index.php(19): yii\base\Application->run()
#24 {main}

[mark-chief]

@robzor There is a dump of HTML that I cant really make anything of..

[robzor]

@mark-chief I can see a commit 1 hour ago so I think @kringkaste is busy working away trying to help us :)

[kringkaste]

We have released a new version 2.2.1, that should fix the issues using the proxy. The release 2.2.0 has only worked correctly for all accounts/tags, when not using the proxy. The 2.2.1 addresses the issue, that we get different responses from Instagram.

Thank you all for your patience and your contribution. This is really awesome.

Update the plugin, invalidate your caches now, and if you use the proxy, give him some time to update the data for you.

And of course, if something is still not working, please give use the account/tag you try to get. Then we can have a look into it and will help you asap.

Craft3: We will now backport the 2.2.1 to 1.4.0 in the next hours.

[robzor]

We have released a new version 2.2.1, that should fix the issues using the proxy. The release 2.2.0 has only worked correctly for all accounts/tags, when not using the proxy. The 2.2.1 addresses the issue, that we get different responses from Instagram.

Thank you all for your patience and your contribution. This is really awesome.

Update the plugin, invalidate your caches now, and if you use the proxy, give him some time to update the data for you.

And of course, if something is still not working, please give use the account/tag you try to get. Then we can have a look into it and will help you asap.

Craft3: We will now backport the 2.2.1 to 1.4.0 in the next hours.

Thank you so much, testing now! :)

[mark-chief]

@kringkaste Unfortunately it is not working for me. I am no longer getting that error, the feed just doesnt load. Anything I can debug for you?

[kringkaste]

@mark-chief Can you give me the accountname / tag you try? Do you use the proxy?

[mark-chief]

@kringkaste The dump is loading the profile information only, and no posts. (Edit: i also use the proxy)

[robzor]

Same here, it's not loading on my local environment.

Using proxy.

[kringkaste]

Yes that's the old data from the proxy, I updated your data manually, you now should get the new version.

[kringkaste]

@robzor Same, I updated your proxy data manually, should work now.

[mark-chief]

Thank you @kringkaste , really appreciate your ongoing efforts with this plugin!

[robzor]

@robzor Same, I updated your proxy data manually, should work now.

That's working on staging now, thank you! :)

[jamesmacwhite]

@kringkaste Thanks for the work on the updating the plugin and proxy. I can confirm that the proxy response now returns the correct data.

Currently the Craft CMS 3 version of the plugin won't directly return data for templates however, due to the parser changes required. Is there a rough estimate of a release with the backport will be made for the Craft 3 release? Thank you!

[kringkaste]

Hello to all who are running Craft 3: We just released version 1.3.0 of the plugin for Craft3. Could you please update, invalidate your data cache (not only the Instagram data cache) and check if it now works for you?

[laurabennett]

Working for me. Thanks!!

On Thu, Jan 11, 2024 at 9:29 AM Tom Schulze @.***> wrote:

Hello to all who are running Craft 3: We just released version 1.3.0 of the plugin for Craft3. Could you please update, invalidate your data cache (not only the Instagram data cache) and check if it now works for you?

— Reply to this email directly, view it on GitHub https://github.com/codemonauts/craft-instagram-feed/issues/70#issuecomment-1887307099, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYJKJRR7O72FUNYKYYRWJTYN7ZLTAVCNFSM6AAAAABA6C7LD2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXGMYDOMBZHE . You are receiving this because you were mentioned.Message ID: @.***>

--

Laura Bennett digital diva

drive > P.O. Box 2838, 170 Kearsarge Street North Conway, NH 03860 603.356.3030 @.***

[jamesmacwhite]

Also confirmed working for me now through the plugin for use in templates. The igproxy.codemonauts.com proxy was returning the right data since a few days ago, but the parsing side was out dated.

Thanks again for the work!

[laurabennett]

This was working on Craft 3 but now seeing "Unknown Account" and no images.

[angelokeirsebilck]

This worked again after updating the plugin on Craft4. But now also not seeing posts but seeing 'Unknown Account'.

[mark-chief]

Also seeing unknown account on Craft 4

[kringkaste]

We have solved the problem. All accounts that were affected should receive data again in the near future. Please clear the Instagram data cache if not. The reason was a wild running proxy server. Sorry for the inconvenience.

[deevanych]

We have solved the problem. All accounts that were affected should receive data again in the near future. Please clear the Instagram data cache if not. The reason was a wild running proxy server. Sorry for the inconvenience.

still not works ( instagram feed 2.2.1, craft cms 4.5.13

dd(craft.instagram.getFeed()) returns []

[kringkaste]

@deevanych Yes, it now returns an empty array but not the dummy data for "unknown account". We are updating the affected account at the moment. You should get the data soon, the queue is beeing processed.

[laurabennett]

Also seeing an empty array on our end. Thanks for the update.

On Mon, Jan 22, 2024 at 6:11 AM Tom Schulze @.***> wrote:

@deevanych https://github.com/deevanych Yes, it now returns an empty array but not the dummy data for "unknown account". We are updating the affected account at the moment. You should get the data soon, the queue is beeing processed.

— Reply to this email directly, view it on GitHub https://github.com/codemonauts/craft-instagram-feed/issues/70#issuecomment-1903780396, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYJKJVR6MK2SRJYPWNBYNLYPZCORAVCNFSM6AAAAABA6C7LD2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBTG44DAMZZGY . You are receiving this because you were mentioned.Message ID: @.***>

--

Laura Bennett digital diva

drive > P.O. Box 2838, 170 Kearsarge Street North Conway, NH 03860 603.356.3030 @.***

[robinsm]

We are no longer receiving the latest posts from the account we are querying in case this is a related issue.

[laurabennett]

Looks good now. Thanks!

On Mon, Jan 22, 2024 at 9:44 AM Robin Mannering @.***> wrote:

We are no longer receiving the latest posts from the account we are querying in case this is a related issue.

— Reply to this email directly, view it on GitHub https://github.com/codemonauts/craft-instagram-feed/issues/70#issuecomment-1904153567, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYJKJVOHE635JO5RPV3CCTYPZ3OXAVCNFSM6AAAAABA6C7LD2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBUGE2TGNJWG4 . You are receiving this because you were mentioned.Message ID: @.***>

--

Laura Bennett digital diva

drive > P.O. Box 2838, 170 Kearsarge Street North Conway, NH 03860 603.356.3030 @.***

[deevanych]

still returns emtpy array 😥

[mark-chief]

As above, not seeing any posts. Logs show:

2024-01-23 08:29:52 [web.ERROR] [instagramfeed] Error fetching page: Client error: `GET https://igproxy.codemonauts.com/removedUsername/` resulted in a `404 Not Found` response: Not found  {"memory":10027568} 
2024-01-23 08:29:52 [web.ERROR] [instagramfeed] Instagram profile data could not be fetched. {"memory":9987536} 

[sonoflee]

Also not seeing any posts on Craft 3 installation after updating to v 1.3.0 and clearing all caches - getting the same error as above: Error fetching page: Client error: 'GET https://igproxy.codemonauts.com/[username]/' resulted in a '404 Not Found' response: Not found

[kringkaste]

Hey @sonoflee @mark-chief @deevanych @robinsm The queue is very long at the moment and is still processing. We would love to prioritize you. Please send us a short mail to support@codemonauts.com with the account names or tags affected.