Internal Server Error when attempting Delete

[irdublu]

This is a web site based on SimpleSite with MailKit and SiteMap plugins.

When in admin, I am getting Internal Server Errors when I try to delete some items such as a page or an image. The error messages are different as is the stack trace, that I don't know how to diagnose. I've looked at the delete stored procedure but find it very complex to unravel and understand.

The db seems to work Ok, in that the site loads pages and their content. My really big worry is the db is corrupt in some way so, making a new one and copying the content over won't clean things up. I don't want to have to start again completely as there is over a month of work here.

Would you be able to help diagnose the stack trace or guide me as to a safe way to clean up the db? If so, I can post images of the error messages (2), stack traces (2) and data in tables (whatever you deem necessary).

When investigating, you may want to know that I found that when I filter the error log search, the search box is so offset that, even when the screen is maximised, you can't see what is input. I attach an image here where I deliberately typed more characters just to get at least one to show for this illustration. You can see the centre-aligned button text is no longer appearing to be central..

[HeyJoel]

Error messages/stack traces would be a good start. Perhaps there is some dependency that is preventing deletion that I have missed.

[irdublu]

These are the details relating to trying to delete a page

Immediately after clicking the Delete Confirmation 'Yes' button, it starts with this

---------------------------------------------------------------------------------------------------------------------- The Error tab shows this item

----------------------------------------------------------------------------------------------------------------------- and when clicked, this

-----------------------------------------------------------------------------------------------------------------------

These are the details relating to trying to delete an image

Immediately after clicking the Delete Confirmation 'Yes' button, it starts with a dialogue similar to above

The Error tab shows this item

----------------------------------------------------------------------------------------------------------------------- and when clicked, this

-----------------------------------------------------------------------------------------------------------------------

I will appreciate any help you can offer.

[HeyJoel]

Before I dive in, can you just confirm your Cofoundry and SQLServer version please?

[irdublu]

Hope this is what you need Cofoundry: 0.10.1 SQL Server: 2019 v15.0.2000.5 SSMS: 2018 (I think) v18.11.1 Visual Studio: 2022 v17.0.4

[HeyJoel]

I can't seem to replicate either of these errors, so I'll need a little bit more information.

1) For both errors, does it occur on a particular page/image, or all, e.g. can you create and delete a test image/page. I've tried adding various relations and dependencies, but maybe there's a particular relationship that is causing the issue. 2) Page: Can you check the page deletion trigger exists in your database? You can do so with this query: select * from sys.objects where [object_id] = OBJECT_ID(N'Cofoundry.[Page_CascadeDelete]') and [type] = 'TR'. If not, do you have any errors in the Cofoundry.ModuleUpdateError table? 3) Image: I'm trying to pin down the statement that is causing this, I have a suspicion, but I can't replicate it in my environment. Can you confirm you are targeting .NET 3.1? The issue may be a plain Linq statement that is behaving differently for you than for me.

[irdublu]

1 I've just tried deleting another page and it fails. I have just created a new page successfully but it failed to delete.

2 a. I ran your script against my db. It produced a list of column names but no records. I've looked in tables, stored procedures and functions can not see it anywhere. It looks like it is a trigger? If so, I have no triggers but can't think how the script returns column names.

2 b. Yes, I've made this discovery just before your reply came in. There are 68 lines of errors. All relate to 00009sql a script I can not find. I have scripts 00001 - 00008.

I started with SimpleSite (probably version 9 to start with) and its InitData, then added Menus and its InitData then added the SiteMap plugin and finally the Mail plugin. Could I have confused the db as I see many errors are saying duplicate object cannot be created.

Do you want to see a few sample lines that are different types of error from the Cofoundry.ModuleUpdateError table?

3 Yes definitely 3.1. ```

netcoreapp3.1```

[HeyJoel]

It looks like there was an issue applying the latest update. Can you provide the error from the log?

Those init data scripts from different samples aren't meant to be combined, so I don't know offhand what effect that would have, but yes perhaps that would cause a data issue. I'm sure we can sort it though. Did you get any errors running those scripts?

[irdublu]

If you mean an error generated when upgrading from Cofoundry 0.9.0 to 0.10.1 with the Nuget package, then as far as I can tell, the upgrade went fine and I can't see one listed in the Admin log tab.

When starting out I needed the EntityDefinitionsfor the menus and reckon this is when I may have screwed up the SimpleSite page IDs amongst other things.

As for getting errors running the scripts, I can't remember as it was a while back, but I think the answer is 'yes'. Unsure of any significance and the site appeared to work, so I continued development. Nothing seemed wrong until I tried deleting objects, such as images and pages, and couldn't.

This web site is one where I had the original design and scripted the schema and content using db Tasks Generate Script into a new site. I have done that twice now. I still have both scripts. but have changed the site a fair bit since.

[HeyJoel]

I'm looking for the errors in 'Cofoundry.ModuleUpdateError'.

[irdublu]

There are 68 lines so I removed the ModuleUpdateErrorId and ExecutionDate as all were just microseconds apart then used DISTINCT. File attached. My growing opinion is the damage was done in the previous db before I scripted it into this db. Cofoundry.ModuleUpdateError.txt

[HeyJoel]

Thanks, I was expecting an error with creating the triggers, but there seems to have been an issue with installing the update which will have since been resolved (otherwise the site would not run).

I think the best next step is just to try and create/recreate all the stored procs/triggers etc to ensure you have all the latest objects installed, and that they can be installed without error. I've produced a script that combines all these for Cofoundry version 0.10.X here, can you run that against your database and let me know if you receive any errors? The script does not change any data, but best backup your database before you run it anyway.

[irdublu]

I've copied your script and looked through some of it to see if it is db specific. Will it affect the other Cofoundry dbs I have such as Mail, PageBlockTypes, Menus and SimpleSite? Should I just head the script with USE [dbName]?

[HeyJoel]

The script is db specific, just point it to the database you're having trouble with.

[irdublu]

Thanks. All done. I have your 5 sample images and a load of mine in the App_Data folder and loaded into the web site. I have managed to delete No 5 Manny from the Admin list of Images. I tried to delete another of the sample images and get the same error as before. New images attached:

---

---

I do seem able to create and delete pages now. I have just reloaded Manny image 5 and deleted it again successfully. The other images won't delete.

I'm concerned to see there are now 1052 errors listed. I'm wondering if there is a way to identify my page, image and menu content in this db (I have no documents and just one other user so far), make a new db then copy these specific items of content to the new db so it is clean but it means I don't have to recreate all my pages and menus. This experience has left me wondering that if content does become corrupt in the db, is there some way of starting afresh without having to completely rebuild the site from scratch.

[HeyJoel]

I think there's a few things going on here:

1) The triggers weren't installed into your database - I had a suspicion that this might have been to do with your "generate scripts" method of copying the database, and having tried it myself it seems the default option has "Script Triggers" set to false. No idea why. I would instead use "Export Data-tier Application" to export to a .bacpac file. 2) There is an issue deleting the images generated from the sample data import scripts, I can now replicate this, but I'll look into this further tomorrow. It's something to do with the dependency checking, as the images are being used in pages and page modules. If the error did not occur and the dependency check was working as intended, it would warn you that you cannot delete the image because it is in use. If you remove the dependencies (e.g. delete the page or swap out the images) then you will be able to delete the image. I don't think the data is necessarily corrupt, I think it's just a particular set or ordering of dependencies that I have not been able to replicate until now.

With regards to the "1052 errors", is this a growing number? If the site has started then it means the auto-update has completed. When the auto-update process encounters an error, it will wait a short period and try again, so while it's unusual to encounter an error during installation, if you do encounter one there can be a lot of error messages as the process continues to try and run the update (with an exponential backoff). If the errors are continuing to occur while the site is running fine, then that certainly is odd, and maybe you have another instance running against the database that is misbehaving?

[irdublu]

1. Yes, whilst I had looked through the script parameters, I missed the fact Triggers were set to false. Did a test .bacpac and will use in the future. Thank you for that.
2. I'm getting this on my images too, not just your sample images.
3. Error log. I've run the site today and the figures '1052' no longer appear on the Errors page and it is back to the 59 errors that it was before and no longer displays the number of errors. There is something strange about the display. Page 1 is fine but page 2 does not understand the viewport. I attach two files to illustrate this. Maybe you can tie this in with the image I sent when trying to filter errors in my opening post on this issue.

   ---

   

   ---

   

   ---

   Even though the top bar to illustrate how much more can be viewed shows a little bit more can be seen, even when the screen is maximised, only a small part of the url is visible and User Agent and Date are always hidden. Is this just my installation or have I found something new? Do you want these matters raised in a separate Issue to keep this one just on data deletion issues?

[HeyJoel]

Yes, that is a separate issue, I've pushed it out to #508.

The image problem is a generally issue with dependency checking in some scenarios, but if you're getting the error, it means it is a required dependency to other entities. This is a domain layer bug that I will get fixed, and isn't an issue with your database.

[irdublu]

That's a relief. I was so close to starting a brand new site and db and wondering how to get all my content in. I still see that as a possibility, since I screwed up by installing two lots of InitData from different sample projects and am concerned something might still be wrong.

I must say thank you for your time on this, over the weekend too - very much appreciated and great support.

I'll await something to show the bug fixed.

[HeyJoel]

The fix has now been released in v0.10.2.

It's hard to say without going through your data with a fine tooth comb, but I wouldn't worry too much about the "InitData" causing problems. These scripts use identity insert to add in a handful of sample objects such as pages and custom entities, so at most the problems will be limited to those entities where you may have an entity from one sample referencing one from another. In any case, you can delete these sample entities, or fix any incorrect references you find. It shouldn't affect any new data you add.

[irdublu]

v0.10.2 installed. No error occurs now and a new message appears saying Cannot delete this image because page '*******' has a dependency on it. This is true - the image is in a page so I left Admin, edited the page to change the image to another and saved. Went back to Admin Images and tried to delete the image but it insists Cannot delete this image because page '***' has a dependency on it.` which is no longer true. Closed the site down, re-opened it and still the same situation.

I tried to diagnose this myself but cannot yet understand which table ties an image to a page.

[HeyJoel]

Is this one of your pages or one of the ones imported by the InitData script?

[irdublu]

One of mine. It is in a split content PageBlockType of my making that copies your design but changes the ratio from 50:50 to 75:25.

[HeyJoel]

When you removed the image, did you publish? If the image is referenced in the published or draft version it will be seen as a dependency, if it is in a previous version it should be ignored.

I've tried to replicate, but it appears to be working in my tests, I've tested references in old versions and on removed regions, those are the two scenarios that I could think of that might trigger an issue. The other scenario I was thinking of for you would be if the references was adding in via the "InitData" script, if you had run this after you had created the page then it may add the dependencies to your page rather than the intended.

You can see the raw list of relations by running this query, you can get the image id from the URL in the admin panel:

declare @ImageAssetId int = 9

select u.RootEntityDefinitionCode, e.[Name] as RootEntityName, u.RootEntityId
from Cofoundry.UnstructuredDataDependency u
inner join Cofoundry.EntityDefinition e on e.EntityDefinitionCode = u.RootEntityDefinitionCode
where RelatedEntityDefinitioncode = 'COFIMG' and u.RelatedEntityId = @ImageAssetId and RelatedEntityCascadeActionId = 1

Cofoundry uses this list to determine if the image can be deleted, however it will also check to ensure that the relation isn't related to a previous version.

[irdublu]

Thanks. The page had been published but is unpublished again at the moment.

I think it highly unlikely I added any 'InitData' once I had created my own pages and images but I do recall adding Image Id 5 Manny to the Admin list for some reason., probably as a test for something.

I see the image I am trying to delete (Manny) has two entries in the Windows Image folder; Image Id 5 and Image Id 26. Only the image Id 5 appears in the Admin Image list and it is that one I am trying to delete.

I ran your code, did some digging and see image Id 5 produces no records and Image Id 26 lists one Page Version Block record. In the ImageAsset table Image Id 5 does not exist any more but Image Id 26 does. I haven't been able to work out to what object RelatedEntityId refers. It states the Id is 26. It may be an unfortunate coincidence but the PageBlockType I created has a PageBlockTypeId of 26.

Even though the image is referenced to my page, it does not appear on it.

I went to the page in question and in Edit mode, deleted the PageBlockType that once displayed my text and the image and replaced it with a Content Block, so it is now text only. Then, when trying to delete image Id 5, it still claims to be referenced to my page. I closed the site and reloaded it to refresh the db but no improvement.

As you would expect from your code, the RelatedEntityCascadeAction for image to be deleted is set to None.

I am reluctant to delete anything manually from folders or tables without guidance in case I cause some corruption but maybe this is the only solution. If so, can you advise a safe way of doing this please?

[HeyJoel]

So to clarify, you say:

Only the image Id 5 appears in the Admin Image list and it is that one I am trying to delete.

but then say

In the ImageAsset table Image Id 5 does not exist any more but Image Id 26 does

If the image asset appears in the admin list, it should be in the database - there's no caching on that query so what you see should be what's in the database. Is it possible the image is titled "5", in that perhaps you re-uploaded the original asset file named "5.jpg" as a new asset, which is now titled "5" but actually has an ImageAssetId of 26?

The ImageAssetId can be found in the admin panel in the details url in the format "/admin/images#/{ImageAssetId}", so for an ImageAssetId of 5, the admin details URL will be "/admin/images#/5". Unfortunately the ImageAssetId isn't actually displayed in the UI.

[irdublu]

Yes, I see Image labelled 5 is in fact Image Id 26; thank you for that tip on discovering this.

Your code above finds this record: COFPGB Page Version Block 58

I then did a Select on Table UnstructuredDataDependency WHERE RootEntityId = 58 and it found this record

RootEntityDefinitionCode, RootEntityId, RelatedEntityDefinitionCode,    RelatedEntityId, RelatedEntityCascadeActionId
COFPGB, 58, COFIMG, 26, 1

When I search the db for 'RelatedEntityId' it seems no other table in the db has this column so I don't yet understand its significance. All I find are 4 errors in the CofoundryPlugin.Error table. Three are the error in the attached image (or very similar). One is different. They are so long I haven't found a way to output them to a text file for you.

To the best of my knowledge, I have no Page or other object that uses Image Id 26 (there is a PageBlockType with that Id), yet still cannot delete it. I have tried some T-SQL to search for what might still be using Image Id 26 but find no clues. Is there some way of doing this?

---

[HeyJoel]

Thanks for this, firstly now that we know it's ImageAssetId 26 I can be fairly sure this isn't to do with the InitData issue, but I still can't replicate this in my own environment. So, to find out more information about how this is linked, can you run this query?

select 
v.PageId, pdp.FullUrlPath + '/' + p.UrlPath as [URL], v.Title, p.PublishStatusCode, v.PageVersionId, v.DisplayVersion, wfs.[Name] as VersionWorkFlowStatus,
t.[Name] as TemplateName, t.IsArchived as IsTemplateArchived,
r.PageTemplateRegionId, r.[Name] as RegionName, r.IsCustomEntityRegion,  
b.PageVersionBlockId, b.SerializedData, b.CreateDate, b.UpdateDate 
from Cofoundry.PageVersionBlock b
inner join Cofoundry.PageVersion v on b.PageVersionId = v.PageVersionId
inner join Cofoundry.PageTemplateRegion r on r.PageTemplateRegionId = b.PageTemplateRegionId
inner join Cofoundry.PageTemplate t on t.PageTemplateId = r.PageTemplateId
inner join Cofoundry.WorkFlowStatus wfs on wfs.WorkFlowStatusId = v.WorkFlowStatusId
inner join Cofoundry.[Page] p on p.PageId = v.PageId
inner join Cofoundry.PageDirectoryPath pdp on pdp.PageDirectoryId = p.PageDirectoryId
where b.PageVersionBlockId = 58

This should tell you exactly which block, version and page the image is linked to. I'm particularly looking to understand if it is a previous version, what the workflow status is, what the publish status is etc (P = pubished, U = unpublished), if the template is archived (1 = true / 0 = false) etc.

If you're sure it's not being used then there is no harm in deleting the reference in the UnstructuredDataDependency table - currently this is only used for dependency checking for deletions, but I'd really like to understand where the issue is so I can get it fixed.

[irdublu]

I ran your code and attach a text file with the output. It is Unpublished, the VersionWorkFlowStatus is Published, not Archived. 58.txt

I show the Page in Admin and that it has three Content Sections

and as Content Sections can include images, I show the page in Preview to witness no images appear. They did, but as mentioned earlier, I removed the images and changed the Split Content Block Types to Content Block Types.

[HeyJoel]

Ok, I see it now. The reason you're getting that message is because you have not published the version where you removed image 26. A dependency still exists on the "unpublished" V2 version, and this constraint is enforced because if you discarded your draft, V2 would become the latest version, and you would have a broken dependency.

So, those dependencies are released once your changes are published.

I think it's right to hold that dependency, but I guess this can be confusing, moreso because the editor does not allow you to see V2 as the "previously published" version, and also because the dependency message doesn't explain that either.

Does that make sense?

[irdublu]

When I read your words on Monday

"When you removed the image, did you publish? If the image is referenced in the published or draft version it will be seen as a dependency,"

Rightly or wrongly, I took it to mean I should not have published. The page was published, so I unpublished it. Maybe, crucially, I did not re-publish it.

Reading what you now say, I have published the page again and successfully deleted Image Id 26. Joy!

I get the idea that old dependencies are released upon publishing but have yet to get to grips with versioning. I'll read your Docs page on Entity Versioning again but my understanding first time round wasn't enough to get me out of this situation. Again, thanks for your persistence in resolving this.