App Issue: "Views" disappear and prompt for new choice

[tpperlman]

I'm submitting a ... [x] bug report

...about [x] admin experience UI [x] Razor templating

Current behavior Some page modules randomly disappear on the page, and when we login as an admin we are prompted to choose a view for an existing app on the page. The views do "reappear" in the admin panel when we create a new one using the same Razor template, but after an app recycle they randomly disappear again. It isn't always the same app/view either. Sometimes one disappears, and another on a different recycle. After a couple recycles, we can see the views back and choose them again for the module on the page, but after clearing cache/recycling they tend to disappear again. We had been running 2SXC v12.08 and upgraded to see if it corrected the issue but it persists.

Instructions to Reproduce the Problem We don't necessarily know the cause, seems to be caching related, but one app uses a data from a query and another doesn't require any data.

Your environment Windows ASP.NET

• 2sxc version(s): 12.10
• Browser: Any
• DNN: 09.10.02
• Language: Any

[iJungleboy]

This is super-odd, never heard of it before.

Could it be related to the Dnn version? You are using a very new Dnn ad the selected view is actually stored in Dnn Module Settings.

[tpperlman]

@iJungleboy it is weird indeed. I will email you a video link demonstrating the issue.

We usually do a build out of new sites on the latest version and if we run into breaking issues we downgrade.

[iJungleboy]

@tpperlman Please do check if it happens on an earlier version, because this is to 99% not a 2sxc issue, but Dnn data/settings caching.

I'll close for now, but do post again if something indicates it's a 2sxc issue.

[tpperlman]

@iJungleboy We have another DNN v9.10.2/2SXC v11.22.0 site and haven't been having any issues. We can't feasibly downgrade the DNN site to an earlier version.

When we open the 2SXC admin panel for the 2 apps that are exhibiting the issue, we can see in the browser's development tool's Network tab that the API calls return no data when the view disappears:

ENDPOINT: /api/2sxc/admin/view/all?appId=6 RESPONSE: []

...and when we create a new view the original one re-appears and this is the response from the same endpoint:

ENDPOINT: /api/2sxc/admin/view/all?appId=6 RESPONSE:

[{"Id":173,"Name":"Instagram","ContentType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"PresentationType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"ListContentType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"ListPresentationType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"TemplatePath":"_instagramFeed.cshtml","IsHidden":false,"ViewNameInUrl":"","Guid":"d2a37b3e-aba4-4c04-9ed4-9210d64ef4dc","List":false,"HasQuery":false,"Used":0,"IsShared":false,"Permissions":{"Count":0}},{"Id":191,"Name":"test","ContentType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"PresentationType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"ListContentType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"ListPresentationType":{"StaticName":"","Id":0,"Name":"no content type","DemoId":0,"DemoTitle":""},"TemplatePath":"_instagramFeed.cshtml","IsHidden":false,"ViewNameInUrl":"","Guid":"477f9577-49e1-4b51-9c41-3b870201e6d4","List":false,"HasQuery":false,"Used":0,"IsShared":false,"Permissions":{"Count":0}}]

Why would calls to the same 2SXC endpoint return different data? Has there been a change to 2SXCs caching?

Further, when I look at 2SXC Insights at /api/2sxc/sys/insights/entities?appid=6&type=2SexyContent-Template for the template entities I see this:

Entities for 2SexyContent-Template (2SexyContent-Template/2SexyContent-Template) in 6

entities: 3
#   Id  Guid    Title   Type    Modified    Owner   Version Metadata    Permissions
1   173 d2a37b3e-aba4-4c04-9ed4-9210d64ef4dc    Instagram   2SexyContent-Template   11/22/2021 16:27:49 dnn:userid=1    2   0   0
2   192 c5e0ff5f-b784-40d8-b5be-6665e567653b    Test    2SexyContent-Template   12/06/2021 15:38:53 dnn:userid=1    1   0   0
3   193 ce6770df-dfe1-47f6-9b19-9873e904d295    test2   2SexyContent-Template   12/06/2021 15:41:00 dnn:userid=1    1   0   0

Then when I recycle the app and the view disappears I see this refreshing the same insights screen:

Entities for 2SexyContent-Template (2SexyContent-Template/2SexyContent-Template) in 6

entities: 0
#   Id  Guid    Title   Type    Modified    Owner   Version Metadata    Permissions

[iJungleboy]

2sxc asks DNN using the ModuleId for the Settings which tell it what app, content-block etc. If this fails (because maybe DNN module-settings caching is broken), then the result is empty and the behavior will be completely off.

Please check the DB in the Dnn Module Settings to see if the settings are there (or maybe they are duplicate?) - compare with a system which works. You should usually have the App name/id and a content-block reference. The template reference can also exist for not-yet-initialized modules (where you're still switching views)

Note that if this only happens on one single module, I suggest you delete it and create a new one.

[tpperlman]

We understand about how the DNN module stores the view for an app, and we did delete the affected modules, emptied the recycle bin and recreated them. The data is still going missing when hitting the 2SXC API directly in insights:

We just tested this 2SXC API again at /api/2sxc/sys/insights/entities?appid=6&type=2SexyContent-Template:

Entities for 2SexyContent-Template (2SexyContent-Template/2SexyContent-Template) in 6

entities: 3
#   Id  Guid    Title   Type    Modified    Owner   Version Metadata    Permissions
1   173 d2a37b3e-aba4-4c04-9ed4-9210d64ef4dc    Instagram   2SexyContent-Template   11/22/2021 16:27:49 dnn:userid=1    2   0   0
2   194 0742dad7-1f5a-409f-b17a-5bdd8fe7f5cd    test    2SexyContent-Template   12/06/2021 15:48:19 dnn:userid=1    1   0   0
3   195 8a84caec-5c87-45e3-8fea-536ac3dd9a44    test    2SexyContent-Template   12/07/2021 09:25:51 dnn:userid=1    1   0   0

...then recycled the app and see this:

Entities for 2SexyContent-Template (2SexyContent-Template/2SexyContent-Template) in 6

entities: 0
#   Id  Guid    Title   Type    Modified    Owner   Version Metadata    Permissions

We took the module out of the equation, which means something must be going on in the caching of entities.

It takes us creating a new duplicate view in the app admin to get the cache to refresh. Then sometimes when we delete one of the new duplicate views they all disappear again in the app admin.

[iJungleboy]

This is weird, and bad. I'll reopen till we have a better insight.

Are you by any chance using a farm, or have your IIS configured to use multiple threads? That's often a mis-configuration which results in stand-alone programs running parallel - each with a different cache.

[iJungleboy]

@tpperlman This is important to me. Is there any chance I could get some remote access? I would need the login details + host access...

[tpperlman]

@iJungleboy We're not doing anything out of the ordinary, this is a site at a hosting company with multiple DNN installations on Plesk of same/various versions. No special configuration changes in this pool. This is the only DNN install running a 2SXC v12 version however.

I'll follow up via email with credentials you can use to review the site, and thank you for your help!

[iJungleboy]

I've done some checks and I believe it's somehow related to the Search Indexer. We've seen a similar problem a while ago on https://dnncommunity.org/ but didn't find time to look deeper.

The problem appears to work as follows:

1. If the search indexer runs right after restart, it will ask 2sxc for data
2. Then 2sxc tries to load the data from the DB, but doesn't get all the information from Dnn in regards to languages etc. so something seems to be off. The problem is probably a mismatch between the languages list which DNN provides during search-indexing and the languages list later on at runtime
3. On the other hand, if the indexer doesn't run first, the data is already loaded when the indexer runs and everything is as expected

We'll prioritize this - @tvatavuk could you have a look?

For an workaround, you could disable search or set it to a much slower interval, so the chance is much smaller that it's the first thing to load the data.

[tpperlman]

@iJungleboy thank you so much for helping research this.

We took your advice as a test and disabled the Search: Site Crawler task in the scheduler. Unfortunately after a couple recycles the 2SXC app views disappear again. After a couple subsequent recycles they reappear and we can select them. Then after a couple more recycles, they disappear again.

We hope this helps in your diagnosis.

[iJungleboy]

I have found another similarity to the dnncommunity sample, let me check...

Where could I find App 4 BTW, 6 is the instagram feed, which is #4 which also causes trouble? because I have a hunch...

[iJungleboy]

Still working on it - I'm still not 100% sure, but I believe it has something to do with Dependency Injection scopes...

[tpperlman]

Thank you for the update! App ID 4 is the Home Hero Video/Slider.

[iJungleboy]

I finally found and fixed it 😁

Here's what's happening:

1. The app is very small - it has exactly 3 entities which describe the view
2. This meant that in some cases loading was so fast, it took less that 1/100th of a nanosecond - so less than 10'000th of a second
3. I believe this was most common when the search indexer ran, because in that case there was no HttpRequest and everything was faster than usual
4. Basically this meant that after loading the cache-timestamp was identical with the initial cache timestamp, so anything upstream didn't realize that the data had changed

To fix it, I modified the app-state loading to start with a timestamp of -1/100 nanosecond (10 picoseconds). So after loading the cache timestamp will be off by at least 1 and everything knows about the changes. Here's a before/after picture of the loading logs:

@Tychodewaard This is also what's causing the same issue on DnnCommunity. It will be fixed in 2sxc 13 which should be released in a few days.

[tpperlman]

Wow, give @iJungleboy a bier! Fantastic detective work!! Proost!

We restored our site from backup, re-installed your pre-release that was in the /App_Data/ExtensionPackages folder and can confirm after a bunch of clear cache/restart applications that the issue appears to be resolved.

Oh, and don't go picking on the size of our apps :) They always start small, then blow up when the client changes direction 10x. 2SXC is always the best long-term/flexible option.

p.s. As I write this, I'm drinking from my coffee cup that says:

Debugging /De-bugh-ing/ - verb

1. Being the detective in a crime movie where you are also the murderer.

[iJungleboy]

Thanks for the kudos - will cost you at least 10 beers BTW ;)

I was first hunting the wrong suspect for about a day and then needed another 4h to get this resolved. But it was really important, so that's fine ;).

[tpperlman]

We appreciate all you folks do for the community, and in recognition of your efforts we have made a contribution through your 2SIC PayPal account to help offset your time and provide some encouragement! Now, go buy some bier!

Thank you again for the help now and in the future!

[iJungleboy]

@tpperlman that's awesome, thanks!